-- Logs begin at Sun 2022-09-11 14:26:13 UTC, end at Sun 2022-09-11 15:08:54 UTC. -- Sep 11 14:52:52.168385 dv-cinder-856971-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 11 14:52:52.176317 dv-cinder-856971-1-iscsi devstack@keystone.service[81420]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 11 14:52:52.178319 dv-cinder-856971-1-iscsi devstack@keystone.service[81420]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Sep 11 14:52:52 2022] *** Sep 11 14:52:52.178319 dv-cinder-856971-1-iscsi devstack@keystone.service[81420]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 11 14:52:52.178319 dv-cinder-856971-1-iscsi devstack@keystone.service[81420]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 11 14:52:52.178319 dv-cinder-856971-1-iscsi devstack@keystone.service[81420]: nodename: dv-cinder-856971-1-iscsi Sep 11 14:52:52.178319 dv-cinder-856971-1-iscsi devstack@keystone.service[81420]: machine: x86_64 Sep 11 14:52:52.178319 dv-cinder-856971-1-iscsi devstack@keystone.service[81420]: clock source: unix Sep 11 14:52:52.178319 dv-cinder-856971-1-iscsi devstack@keystone.service[81420]: pcre jit disabled Sep 11 14:52:52.178319 dv-cinder-856971-1-iscsi devstack@keystone.service[81420]: detected number of CPU cores: 4 Sep 11 14:52:52.178892 dv-cinder-856971-1-iscsi devstack@keystone.service[81420]: current working directory: / Sep 11 14:52:52.178892 dv-cinder-856971-1-iscsi devstack@keystone.service[81420]: detected binary path: /usr/bin/uwsgi-core Sep 11 14:52:52.178892 dv-cinder-856971-1-iscsi devstack@keystone.service[81420]: your processes number limit is 47680 Sep 11 14:52:52.178892 dv-cinder-856971-1-iscsi devstack@keystone.service[81420]: your memory page size is 4096 bytes Sep 11 14:52:52.178892 dv-cinder-856971-1-iscsi devstack@keystone.service[81420]: detected max file descriptor number: 2048 Sep 11 14:52:52.178892 dv-cinder-856971-1-iscsi devstack@keystone.service[81420]: lock engine: pthread robust mutexes Sep 11 14:52:52.178892 dv-cinder-856971-1-iscsi devstack@keystone.service[81420]: thunder lock: enabled Sep 11 14:52:52.178892 dv-cinder-856971-1-iscsi devstack@keystone.service[81420]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 11 14:52:52.178892 dv-cinder-856971-1-iscsi devstack@keystone.service[81420]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 11 14:52:52.218331 dv-cinder-856971-1-iscsi devstack@keystone.service[81420]: Python main interpreter initialized at 0x5560d9d0f9c0 Sep 11 14:52:52.218331 dv-cinder-856971-1-iscsi devstack@keystone.service[81420]: python threads support enabled Sep 11 14:52:52.218331 dv-cinder-856971-1-iscsi devstack@keystone.service[81420]: your server socket listen backlog is limited to 100 connections Sep 11 14:52:52.218331 dv-cinder-856971-1-iscsi devstack@keystone.service[81420]: your mercy for graceful operations on workers is 90 seconds Sep 11 14:52:52.218718 dv-cinder-856971-1-iscsi devstack@keystone.service[81420]: mapped 403077 bytes (393 KB) for 2 cores Sep 11 14:52:52.218796 dv-cinder-856971-1-iscsi devstack@keystone.service[81420]: *** Operational MODE: preforking *** Sep 11 14:52:52.219099 dv-cinder-856971-1-iscsi devstack@keystone.service[81420]: *** uWSGI is running in multiple interpreter mode *** Sep 11 14:52:52.219099 dv-cinder-856971-1-iscsi devstack@keystone.service[81420]: spawned uWSGI master process (pid: 81420) Sep 11 14:52:52.219305 dv-cinder-856971-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 11 14:52:52.221606 dv-cinder-856971-1-iscsi devstack@keystone.service[81420]: spawned uWSGI worker 1 (pid: 81430, cores: 1) Sep 11 14:52:52.221606 dv-cinder-856971-1-iscsi devstack@keystone.service[81420]: spawned uWSGI worker 2 (pid: 81431, cores: 1) Sep 11 14:52:52.221606 dv-cinder-856971-1-iscsi devstack@keystone.service[81420]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 11 14:52:56.504575 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 11 14:52:56.504575 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: from cryptography.utils import int_from_bytes Sep 11 14:52:56.505369 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 11 14:52:56.505369 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: from cryptography.utils import int_from_bytes Sep 11 14:52:56.505748 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 11 14:52:56.505748 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: from cryptography.utils import int_from_bytes Sep 11 14:52:56.506205 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 11 14:52:56.506205 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: from cryptography.utils import int_from_bytes Sep 11 14:52:56.555620 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 11 14:52:56.555803 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 11 14:52:56.555994 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] command line args: [] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 11 14:52:56.556091 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 11 14:52:56.556211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 11 14:52:56.556297 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 11 14:52:56.556412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] command line args: [] {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 11 14:52:56.556499 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ================================================================================ {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 11 14:52:56.556663 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] admin_token = **** {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.556747 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 11 14:52:56.556854 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ================================================================================ {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 11 14:52:56.556940 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] config_dir = [] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.557083 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.557163 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] admin_token = **** {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.557266 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] config_source = [] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.557599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] config_dir = [] {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.557701 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] debug = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.557780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.557901 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.558204 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] config_source = [] {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.558204 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] debug = True {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.558412 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.558517 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.558645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.558752 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.558876 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.558977 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.559083 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.559190 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.559299 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] list_limit = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.559395 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.559487 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] log_config_append = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.559585 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] list_limit = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.559680 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.559790 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] log_config_append = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.559885 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] log_dir = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.559984 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.560078 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] log_file = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.560183 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] log_dir = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.560268 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.560396 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] log_file = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.560479 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.560550 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.560621 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.560690 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.560811 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.560897 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.561013 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.561094 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.561213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.561294 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.561418 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.561494 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.561615 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.561697 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.561770 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.561892 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.561976 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.562090 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.562171 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.562248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.562373 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.562463 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.562622 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.562707 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.562895 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.563017 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.563172 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.563302 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.563446 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.563657 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.563841 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] publish_errors = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.563917 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.564073 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.564192 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] publish_errors = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.564303 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.564384 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.564506 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.564596 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.564774 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.564856 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.565033 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.565130 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.565262 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] standard_threads = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.565339 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.565495 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.565580 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] standard_threads = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.565728 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.565814 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.565909 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.565986 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.566088 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] use_journal = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.566171 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.566290 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] use_json = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.566363 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] use_journal = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.566438 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] use_stderr = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.566516 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] use_json = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.566591 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] use_syslog = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.566661 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] use_stderr = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.566819 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] use_syslog = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.566900 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.567067 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:52:56.567146 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.567316 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.567398 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.567556 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.567640 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.567802 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.567886 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.568045 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.568128 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.568282 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.568366 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.568526 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.568604 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.568760 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.568836 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.569002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.569079 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.569247 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.569324 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.569496 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.569578 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.569728 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.569806 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.569967 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.570052 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.570200 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.570277 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.570442 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.570526 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.570687 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.570764 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.570948 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.571039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.571187 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.571265 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.571443 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.571525 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.571688 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.571841 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.571916 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.571985 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.572155 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.572232 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.572411 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.572498 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.572661 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.572740 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.572912 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.572992 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.573145 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.573231 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.573377 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.573455 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.573608 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.573685 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.573841 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.573926 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.574076 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.574152 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.574323 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.574399 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.574553 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.574638 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.574784 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.574881 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.575041 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.575117 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.575292 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.575381 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.575527 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.575604 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.575769 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.575844 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.575998 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.576332 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.576423 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.576496 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.576566 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.576635 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.576703 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.576781 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.576889 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.576969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.577121 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.577198 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.577350 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.577498 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.577579 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.577657 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.577810 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.577887 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.578037 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.578123 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.578275 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.578355 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.578506 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.578584 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.578737 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.578870 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.578999 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.579077 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.579235 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.579314 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.579467 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.579552 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.579705 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.579811 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.579943 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.580053 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.580726 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.580874 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.581043 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.581163 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.581279 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.581364 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.581445 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.581517 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.581635 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.581710 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.581879 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.581961 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.582120 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.582203 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.582360 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.582482 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.582596 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.582685 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.582873 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.582998 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.583112 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.583194 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.583346 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.583427 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.583579 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.583697 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.583815 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.583894 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.584073 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.584157 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.584316 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.584393 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.584559 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.584641 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.584795 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.584877 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.585030 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.585107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.585266 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.585346 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.585505 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.585584 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.585753 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.585832 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.585995 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.586075 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.586235 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.586312 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.586469 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.586546 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.586715 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.586793 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.586975 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.587064 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.587212 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.587291 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.587447 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.587525 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.587678 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.587756 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.587914 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.587991 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.588152 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.588232 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.588396 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.588474 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.588645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.588724 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.588886 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.588966 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.589125 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.589209 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.589362 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.589440 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.589587 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.589660 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.589821 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.589901 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.590059 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.590147 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.590292 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.590383 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.590523 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.590601 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.590752 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.590865 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.591011 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.591099 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.591246 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.591325 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.591479 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.591556 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.591713 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.591803 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.591952 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.592036 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.592188 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.592266 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.592432 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.592525 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.592690 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.592770 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.592934 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.593013 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.593168 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] auth.external = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.593256 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.593400 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.593491 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] auth.external = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.593633 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.593712 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.593916 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.593994 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.594148 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] auth.password = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.594234 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.594380 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] auth.password = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.594458 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] auth.token = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.594616 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] auth.token = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.594698 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.594871 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.594968 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.595112 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.595191 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.595342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.595420 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.595582 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.595672 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.596380 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.596512 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.596623 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.596703 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.596861 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] credential.caching = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.596954 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.597109 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.597189 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] credential.caching = True {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.597355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.597434 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.597603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.597690 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.597846 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.597926 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.598091 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.598170 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.598326 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.598412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.598565 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.598642 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.598824 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.598905 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.599074 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.599167 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.599321 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.599400 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.599580 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.599660 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.599830 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.599917 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.600093 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.600181 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.600337 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.600417 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.600592 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.600678 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.600825 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] federation.caching = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.600903 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.601062 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.601142 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] federation.caching = True {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.601301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.601392 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.601544 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.601629 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.601774 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.601854 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.602009 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.602089 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.602244 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.602323 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.602484 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.602562 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.602721 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.602827 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.602993 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.603073 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.603230 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.603308 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.603474 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.603555 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.603708 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] identity.caching = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.603787 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.603949 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] identity.caching = True {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.604027 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.604178 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.604254 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.604416 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.604496 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.604651 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.604737 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.604884 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.604964 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.605115 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.605199 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.605351 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.605429 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.605586 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.605669 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.605827 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.605907 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.606057 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.606136 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.606289 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.606370 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.606520 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.606599 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.606761 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.606887 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.607032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.607114 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.607276 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.607357 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.607522 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.607600 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.607767 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.607846 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.608016 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.608094 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.608258 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.608336 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.608495 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.608575 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.608725 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.608804 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.608957 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.609035 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.609185 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.609268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.609416 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.609496 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.609653 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.609733 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.609888 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.609968 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.610128 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.610206 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.610355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.610433 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.610601 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.610682 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.610868 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.610951 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.611105 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.611197 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.611371 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.611511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.611612 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.611675 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.611835 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.611925 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.612072 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.612151 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.612302 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.612405 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.612541 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.612629 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.612776 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.612856 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.613015 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.613094 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.613251 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.613345 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.613552 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.613632 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.613842 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.613921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.614095 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.614183 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.614329 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.614409 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.614562 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.614642 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.614826 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.614920 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.615084 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.615164 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.615328 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.615416 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.615567 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.615656 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.615806 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.615886 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.616040 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.616120 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.616273 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.user = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.616360 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.616520 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.616600 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.616751 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.616830 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.616999 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.617088 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.617247 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.617327 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.617492 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.617636 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.617714 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.617795 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.617948 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.618028 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.618185 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.618263 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.618416 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.618504 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.618653 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.618731 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.618904 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.618986 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.619138 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.619229 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.619387 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.619465 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.619632 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.619712 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.619906 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.620054 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.620142 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.620217 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.620348 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.620442 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.620590 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.620678 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.620825 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.620904 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.621063 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.621143 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.621301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.621388 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.621535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.621614 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.621771 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.621849 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.622018 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.622106 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.622257 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.622336 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.622496 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.622574 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.622730 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.622838 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.623003 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.623084 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.623238 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.623320 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.623478 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.623587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.623715 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.623809 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.623960 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.624045 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.624199 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.624294 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.624432 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.624519 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.624666 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.624744 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.624904 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] resource.caching = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.624992 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.625147 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] resource.caching = True {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.625226 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.625397 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.625477 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.625633 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.625722 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.625871 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.625950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.626117 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.626196 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.626355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.626436 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.626599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.626678 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.626855 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.626939 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.627609 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.627693 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.627847 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] role.caching = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.628000 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.628179 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] role.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.628254 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] role.caching = True {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.628334 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.628443 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] role.driver = sql {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.628530 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.628678 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.628779 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.628931 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.629019 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.629183 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.629265 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.629436 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.629516 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.629687 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.629775 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.629937 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.630017 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.630187 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.630267 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.630416 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.630503 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.630664 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.630744 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.630930 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.631009 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.631177 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.631269 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.631426 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.631506 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.631673 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.631814 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.631923 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.632051 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.632301 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.632380 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.632542 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.632620 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.632787 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.632873 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.633035 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.633116 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.633272 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.633358 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.633523 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.633603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.633761 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.633841 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.633999 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.634077 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.634232 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.634313 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.634465 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.634545 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.634697 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.634777 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.634955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.635037 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.635190 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.635271 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.635445 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.635524 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.635686 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.635766 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.635924 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.636004 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.636158 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.636238 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.636394 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.636474 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] token.caching = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.636629 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.636710 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.636871 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] token.caching = True {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.636950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.637104 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.637185 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.637356 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.637434 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.637597 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.637676 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.637831 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.637911 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.638077 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.638156 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.638315 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.638393 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.638560 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.638640 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.638798 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.638895 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.639058 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.639138 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.639294 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.639375 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.639532 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.639610 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.639771 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.639851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.640019 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.640100 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.640290 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.640378 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.640522 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.640610 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.640769 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.640850 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.connection = **** {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.641030 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.641118 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.641277 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.641357 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] database.connection = **** {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.641514 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.641595 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.641755 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.641844 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.641991 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.642071 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.642227 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.642308 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.642463 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.642550 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.643408 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.643587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.643664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.643829 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.643910 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.644114 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.644195 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.644357 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.644436 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.644607 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.644686 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.644848 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.644929 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.645090 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.645169 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.645330 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.645408 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.645567 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.645646 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.645803 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.645881 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.646039 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.646117 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.646198 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81430) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 11 14:52:56.646272 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:52:56.646436 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81431) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 11 14:52:56.675237 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.675471 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.675694 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.675785 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.675957 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.676187 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.684277 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.684799 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.713301 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.713525 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.713741 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.713975 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.766498 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.766956 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.815966 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.816198 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.816779 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.817018 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.972681 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.972932 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.973042 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.973454 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.973568 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.973697 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.973802 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.973902 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.974002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.974101 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.974210 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.974210 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.996695 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.996854 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.996947 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.997093 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.997180 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.997318 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.997403 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.997539 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.997631 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.997749 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.997830 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.997963 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.998043 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:56.998178 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:57.009629 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:57.010075 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:57.023035 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:57.023144 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:57.023228 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:57.023371 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:57.023454 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:57.023601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:57.023684 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:57.023825 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:57.023916 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:57.024043 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:57.024158 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:57.024260 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:57.024342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81430) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:57.024476 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81431) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:52:57.041677 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.041890 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 11 14:52:57.042021 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.042235 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.042316 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 11 14:52:57.042442 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 11 14:52:57.042579 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.042717 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.042799 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 11 14:52:57.043005 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.043099 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.043263 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.043347 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.043521 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.043601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.043819 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.053115 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.053396 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.053606 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.053688 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.053890 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.053981 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.054166 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.054453 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.061139 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.061736 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.064055 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.064344 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.064654 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.064791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.064931 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.065071 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.065218 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.065348 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.065458 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.065637 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.065719 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.065927 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.066009 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.066192 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.066272 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.066464 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.066553 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.066739 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.066844 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.067038 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.067316 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.067588 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.090015 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.090724 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.091508 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.091794 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.092212 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.092501 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.097206 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.097492 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.097773 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.098068 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.098356 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.098637 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.104103 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.104394 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.104987 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.105280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.109972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.110272 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.110563 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.110873 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.110959 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.111157 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.111241 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.111479 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.111683 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.111766 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.111985 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.112068 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.112272 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.112556 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.112833 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.113105 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.127484 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.127762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.128710 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.128993 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.131636 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.132831 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.134870 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.135169 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.136058 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.136359 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.139369 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.140514 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.142470 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.144171 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.146202 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.146505 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.146793 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.147102 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.147383 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.147474 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.147660 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.147744 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.147960 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.148245 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.148525 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.148807 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.163303 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.163595 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.163867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.164133 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.164544 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.164839 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.165110 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.165380 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.170881 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.172273 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.172398 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.174387 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.174476 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.174839 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.176192 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.176484 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.177911 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.178201 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.178495 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.178780 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.179167 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.179266 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.179456 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.179742 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.180027 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.180431 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.191154 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.191439 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.191721 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.192001 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.192381 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.192478 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.192672 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.192758 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.192955 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.193039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.193268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.193649 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.193934 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.194227 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.208439 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.209703 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.211374 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.212658 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.214246 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.215702 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.216045 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.217140 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.217518 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.217804 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.218196 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.218614 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.218925 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.219323 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.224875 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.225757 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.226326 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.227247 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.229233 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.229629 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.229918 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.230126 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.230210 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.230524 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.230824 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.231118 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.238481 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.238772 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.239084 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.239606 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.239903 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.240187 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.245946 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81430) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.246233 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.246514 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.246794 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.247090 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.247174 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81431) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:52:57.247373 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.247470 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.247650 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.247735 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.247944 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.248029 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.248199 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.248291 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.248471 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.248553 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.248740 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.248822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.249008 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.249092 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.249275 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.249356 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.249568 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.249651 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81430) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.249850 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.250115 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.250394 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.250667 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81431) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:52:57.284439 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5560d9d0f9c0 pid: 81430 (default app) Sep 11 14:52:57.285740 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5560d9d0f9c0 pid: 81431 (default app) Sep 11 14:52:57.287675 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea5627-f02c-44cc-a97e-6625a6c48df3 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:52:57.287961 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea5627-f02c-44cc-a97e-6625a6c48df3 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:52:57.288229 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea5627-f02c-44cc-a97e-6625a6c48df3 None None] PATH_INFO: `/v3` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:52:57.292396 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 1/1] 10.222.0.33 () {50 vars in 791 bytes} [Sun Sep 11 14:52:57 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:04.194781 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21716b5-8802-4e49-8741-a1ea17e6370e None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:04.195325 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21716b5-8802-4e49-8741-a1ea17e6370e None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:04.195432 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21716b5-8802-4e49-8741-a1ea17e6370e None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:04.199667 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 1/2] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:04.207739 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68312a5-b888-4e81-896d-51831e25dadc None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:04.208306 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68312a5-b888-4e81-896d-51831e25dadc None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:04.208306 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68312a5-b888-4e81-896d-51831e25dadc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:04.289190 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: INFO dbcounter [None req-c68312a5-b888-4e81-896d-51831e25dadc None None] Registered counter for database keystone Sep 11 14:53:04.313897 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG oslo_db.sqlalchemy.engines [None req-c68312a5-b888-4e81-896d-51831e25dadc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81430) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 11 14:53:04.320891 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writer thread running {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 11 14:53:04.511321 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG passlib.handlers.bcrypt [None req-c68312a5-b888-4e81-896d-51831e25dadc None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81430) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 11 14:53:04.511712 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG passlib.handlers.bcrypt [None req-c68312a5-b888-4e81-896d-51831e25dadc None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81430) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 11 14:53:04.570594 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-c68312a5-b888-4e81-896d-51831e25dadc None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:04.572179 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-c68312a5-b888-4e81-896d-51831e25dadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:04.701184 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 2/3] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:04 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 495 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 11 14:53:04.992557 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21716b5-8802-4e49-8741-a1ea17e6370e None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:05.063768 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: INFO dbcounter [None req-f21716b5-8802-4e49-8741-a1ea17e6370e None None] Registered counter for database keystone Sep 11 14:53:05.085848 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG oslo_db.sqlalchemy.engines [None req-f21716b5-8802-4e49-8741-a1ea17e6370e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81431) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 11 14:53:05.092065 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writer thread running {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 11 14:53:05.212265 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21716b5-8802-4e49-8741-a1ea17e6370e None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:05.216616 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072693e0-800b-4cb5-8f57-8d65c16c5131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b748811b1c8a461188c96c2e314bf507', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:05.217257 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072693e0-800b-4cb5-8f57-8d65c16c5131 admin admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:05.217543 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072693e0-800b-4cb5-8f57-8d65c16c5131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:05.217817 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072693e0-800b-4cb5-8f57-8d65c16c5131 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:05.226594 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072693e0-800b-4cb5-8f57-8d65c16c5131 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:05.278904 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: WARNING keystone.common.rbac_enforcer.enforcer [None req-072693e0-800b-4cb5-8f57-8d65c16c5131 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 11 14:53:05.279208 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072693e0-800b-4cb5-8f57-8d65c16c5131 admin admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:05.288937 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 2/4] 10.222.0.33 () {58 vars in 1324 bytes} [Sun Sep 11 14:53:04 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 14:53:05.294877 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68312a5-b888-4e81-896d-51831e25dadc None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:05.312507 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68312a5-b888-4e81-896d-51831e25dadc None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:05.317750 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae915f-f1e8-4da4-b2d1-08978599f4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b748811b1c8a461188c96c2e314bf507', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:05.318901 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae915f-f1e8-4da4-b2d1-08978599f4d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:05.319188 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae915f-f1e8-4da4-b2d1-08978599f4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:05.319471 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae915f-f1e8-4da4-b2d1-08978599f4d6 admin admin] PATH_INFO: `/v3/projects/b748811b1c8a461188c96c2e314bf507` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:05.321082 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ae915f-f1e8-4da4-b2d1-08978599f4d6 admin admin] RBAC: Authorizing `identity:get_project(project_id=b748811b1c8a461188c96c2e314bf507)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:05.373451 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: WARNING keystone.common.rbac_enforcer.enforcer [None req-29ae915f-f1e8-4da4-b2d1-08978599f4d6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 11 14:53:05.373795 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ae915f-f1e8-4da4-b2d1-08978599f4d6 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:05.378990 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 3/5] 10.222.0.33 () {56 vars in 1207 bytes} [Sun Sep 11 14:53:05 2022] GET /identity/v3/projects/b748811b1c8a461188c96c2e314bf507 => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:53:07.499474 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b178b-d601-4b5e-8b07-147fab6566fd None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:07.500038 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b178b-d601-4b5e-8b07-147fab6566fd None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:07.500038 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b178b-d601-4b5e-8b07-147fab6566fd None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:07.504811 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 3/6] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:07.510153 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162b222-584d-4d63-b4a4-624c33e761bc None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:07.510571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162b222-584d-4d63-b4a4-624c33e761bc None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:07.510698 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162b222-584d-4d63-b4a4-624c33e761bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:07.530582 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-5162b222-584d-4d63-b4a4-624c33e761bc None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:07.532147 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-5162b222-584d-4d63-b4a4-624c33e761bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:07.539986 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 4/7] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:07 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 11 14:53:07.811868 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b178b-d601-4b5e-8b07-147fab6566fd None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:07.824635 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b178b-d601-4b5e-8b07-147fab6566fd None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:07.828808 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890b7c4-409a-4cac-a3c0-c99f23513be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b748811b1c8a461188c96c2e314bf507', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:07.829404 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890b7c4-409a-4cac-a3c0-c99f23513be2 admin admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:07.829681 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890b7c4-409a-4cac-a3c0-c99f23513be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:07.830008 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890b7c4-409a-4cac-a3c0-c99f23513be2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:07.837695 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5890b7c4-409a-4cac-a3c0-c99f23513be2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:07.839506 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5890b7c4-409a-4cac-a3c0-c99f23513be2 admin admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:07.848644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 4/8] 10.222.0.33 () {58 vars in 1324 bytes} [Sun Sep 11 14:53:07 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 11 14:53:07.854212 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162b222-584d-4d63-b4a4-624c33e761bc None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:07.864055 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162b222-584d-4d63-b4a4-624c33e761bc None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:07.868440 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ae460-557f-49e3-ac3e-ea6ab52ef549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b748811b1c8a461188c96c2e314bf507', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:07.869068 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ae460-557f-49e3-ac3e-ea6ab52ef549 admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:07.869351 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ae460-557f-49e3-ac3e-ea6ab52ef549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:07.869625 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ae460-557f-49e3-ac3e-ea6ab52ef549 admin admin] PATH_INFO: `/v3/users/53878dbb519c4b3d8858375928f1c2d4` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:07.870978 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042ae460-557f-49e3-ac3e-ea6ab52ef549 admin admin] RBAC: Authorizing `identity:get_user(user_id=53878dbb519c4b3d8858375928f1c2d4)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:07.872845 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042ae460-557f-49e3-ac3e-ea6ab52ef549 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:07.874382 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 5/9] 10.222.0.33 () {56 vars in 1198 bytes} [Sun Sep 11 14:53:07 2022] GET /identity/v3/users/53878dbb519c4b3d8858375928f1c2d4 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:10.114356 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb14e7-2aee-430b-a01a-92db3b604202 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:10.114912 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb14e7-2aee-430b-a01a-92db3b604202 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:10.115016 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb14e7-2aee-430b-a01a-92db3b604202 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:10.116299 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 5/10] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:10.121716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ccce53-9c82-4226-b5d9-abbaacb431ee None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:10.122144 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ccce53-9c82-4226-b5d9-abbaacb431ee None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:10.122281 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ccce53-9c82-4226-b5d9-abbaacb431ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:10.131224 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff2405-9bc7-4a62-86d6-f3f207427283 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:10.131507 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff2405-9bc7-4a62-86d6-f3f207427283 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:10.131780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff2405-9bc7-4a62-86d6-f3f207427283 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:10.133071 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 6/11] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:53:10.138081 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4704ae9-65c2-40ad-8090-7874532d5fbe None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:10.138369 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4704ae9-65c2-40ad-8090-7874532d5fbe None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:10.138644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4704ae9-65c2-40ad-8090-7874532d5fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:10.140894 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-95ccce53-9c82-4226-b5d9-abbaacb431ee None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:10.142098 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-95ccce53-9c82-4226-b5d9-abbaacb431ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:10.181135 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG passlib.handlers.bcrypt [None req-d4704ae9-65c2-40ad-8090-7874532d5fbe None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81431) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 11 14:53:10.181534 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG passlib.handlers.bcrypt [None req-d4704ae9-65c2-40ad-8090-7874532d5fbe None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81431) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 11 14:53:10.225693 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 6/12] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:10.230798 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-d4704ae9-65c2-40ad-8090-7874532d5fbe None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:10.232412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-d4704ae9-65c2-40ad-8090-7874532d5fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:10.332504 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 7/13] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:53:10.495134 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ccce53-9c82-4226-b5d9-abbaacb431ee None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:10.520398 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:10.599349 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4704ae9-65c2-40ad-8090-7874532d5fbe None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:10.602269 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ccce53-9c82-4226-b5d9-abbaacb431ee None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:10.636051 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f77b6b-ac94-4e78-8143-4a4ab1748bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:10.636691 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f77b6b-ac94-4e78-8143-4a4ab1748bab None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:10.636972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f77b6b-ac94-4e78-8143-4a4ab1748bab None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:10.637255 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f77b6b-ac94-4e78-8143-4a4ab1748bab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:10.648316 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f77b6b-ac94-4e78-8143-4a4ab1748bab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:53:10.657166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-e8f77b6b-ac94-4e78-8143-4a4ab1748bab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:53:10.657166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:53:10.657166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:53:10.657166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:53:10.657166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:53:10.657166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:53:10.657166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:53:10.657166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:53:10.657166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:53:10.657166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:53:10.657166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:53:10.657166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:53:10.657166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:53:10.657166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:53:10.657166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:53:10.657166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:53:10.658116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:53:10.658116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:53:10.658116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:53:10.658116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:53:10.658116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:53:10.658116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:53:10.658116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:53:10.658116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:53:10.658116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:53:10.658116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:53:10.658116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:53:10.658116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:53:10.658116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:53:10.658116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:53:10.658116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:53:10.658116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:53:10.659128 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:53:10.659128 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:53:10.659128 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:53:10.659128 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:53:10.659128 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:53:10.659128 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:53:10.659128 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:53:10.659128 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:53:10.659128 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:53:10.659128 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:53:10.659128 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:53:10.659128 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:53:10.659128 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 7/14] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:53:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:53:10.664143 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f77b6b-ac94-4e78-8143-4a4ab1748bab None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:10.700399 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4704ae9-65c2-40ad-8090-7874532d5fbe None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:10.702678 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:10.738339 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c13f0c-c78f-4334-91de-4c46ec1aee60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:10.739234 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c13f0c-c78f-4334-91de-4c46ec1aee60 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:10.739522 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c13f0c-c78f-4334-91de-4c46ec1aee60 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:10.739799 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c13f0c-c78f-4334-91de-4c46ec1aee60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:10.746770 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:10.765592 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f77b6b-ac94-4e78-8143-4a4ab1748bab None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:10.799250 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9dc102-6337-46ba-8a42-a1423c64b1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:10.800007 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9dc102-6337-46ba-8a42-a1423c64b1b3 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:10.800296 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9dc102-6337-46ba-8a42-a1423c64b1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:10.800571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9dc102-6337-46ba-8a42-a1423c64b1b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:10.801236 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9dc102-6337-46ba-8a42-a1423c64b1b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:10.801545 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9dc102-6337-46ba-8a42-a1423c64b1b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:10.802975 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9dc102-6337-46ba-8a42-a1423c64b1b3 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:10.808167 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c13f0c-c78f-4334-91de-4c46ec1aee60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:10.809705 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c13f0c-c78f-4334-91de-4c46ec1aee60 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:10.815487 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 8/15] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:53:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:53:10.821428 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9dc102-6337-46ba-8a42-a1423c64b1b3 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:10.908938 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 8/16] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:53:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 14:53:10.913804 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c13f0c-c78f-4334-91de-4c46ec1aee60 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:10.916547 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:10.920463 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9dc102-6337-46ba-8a42-a1423c64b1b3 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:10.953787 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3d3d4-577e-4b52-b37c-3b80136e80c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:10.954412 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3d3d4-577e-4b52-b37c-3b80136e80c2 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:10.955260 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3d3d4-577e-4b52-b37c-3b80136e80c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:10.955529 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3d3d4-577e-4b52-b37c-3b80136e80c2 None admin] PATH_INFO: `/v3/users/53878dbb519c4b3d8858375928f1c2d4` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:10.956671 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b3d3d4-577e-4b52-b37c-3b80136e80c2 None admin] RBAC: Authorizing `identity:get_user(user_id=53878dbb519c4b3d8858375928f1c2d4)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:10.958132 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b3d3d4-577e-4b52-b37c-3b80136e80c2 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:10.959717 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 9/17] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:53:10 2022] GET /identity/v3/users/53878dbb519c4b3d8858375928f1c2d4 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:53:10.965157 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3d3d4-577e-4b52-b37c-3b80136e80c2 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:10.981670 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:11.015423 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c13f0c-c78f-4334-91de-4c46ec1aee60 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:11.050280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8cec7d-cce0-4fe6-95ad-f0e715630fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:11.051164 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8cec7d-cce0-4fe6-95ad-f0e715630fce None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:11.051444 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8cec7d-cce0-4fe6-95ad-f0e715630fce None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:11.051715 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8cec7d-cce0-4fe6-95ad-f0e715630fce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:11.068447 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3d3d4-577e-4b52-b37c-3b80136e80c2 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:11.070749 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8cec7d-cce0-4fe6-95ad-f0e715630fce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:11.072259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8cec7d-cce0-4fe6-95ad-f0e715630fce None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:11.092163 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-5a8cec7d-cce0-4fe6-95ad-f0e715630fce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:53:11.092163 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:53:11.092163 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:53:11.092163 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:53:11.092163 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:53:11.092163 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:53:11.092163 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:53:11.092163 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:53:11.092163 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:53:11.092163 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:53:11.092163 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:53:11.092163 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:53:11.092163 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:53:11.092163 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:53:11.092163 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:53:11.092163 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:53:11.093196 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:53:11.093196 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:53:11.093196 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:53:11.093196 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:53:11.093196 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:53:11.093196 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:53:11.093196 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:53:11.093196 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:53:11.093196 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:53:11.093196 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:53:11.093196 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:53:11.093196 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:53:11.093196 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:53:11.093196 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:53:11.093196 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:53:11.093196 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:53:11.093196 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:53:11.094008 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:53:11.094008 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:53:11.094008 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:53:11.094008 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 9/18] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:53:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:53:11.099207 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8cec7d-cce0-4fe6-95ad-f0e715630fce None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:11.103544 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a834e-4297-45c3-99a6-1777f05c7535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:11.104181 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a834e-4297-45c3-99a6-1777f05c7535 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:11.104458 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a834e-4297-45c3-99a6-1777f05c7535 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:11.104785 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a834e-4297-45c3-99a6-1777f05c7535 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:11.105942 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8a834e-4297-45c3-99a6-1777f05c7535 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:11.107441 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8a834e-4297-45c3-99a6-1777f05c7535 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:11.108989 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 10/19] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:53:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:53:11.114645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a834e-4297-45c3-99a6-1777f05c7535 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:11.156200 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:11.197690 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:11.204621 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8cec7d-cce0-4fe6-95ad-f0e715630fce None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:11.217444 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a834e-4297-45c3-99a6-1777f05c7535 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:11.239918 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7f866-6cdd-49fe-bef1-8c6ab0f98cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:11.240713 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7f866-6cdd-49fe-bef1-8c6ab0f98cdf None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:11.240991 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7f866-6cdd-49fe-bef1-8c6ab0f98cdf None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:11.241261 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7f866-6cdd-49fe-bef1-8c6ab0f98cdf None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:11.241898 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d7f866-6cdd-49fe-bef1-8c6ab0f98cdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:11.242218 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d7f866-6cdd-49fe-bef1-8c6ab0f98cdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:11.243635 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d7f866-6cdd-49fe-bef1-8c6ab0f98cdf None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:11.252136 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005c939-6f00-4524-ac55-3ab26aeb4fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:11.252885 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005c939-6f00-4524-ac55-3ab26aeb4fb5 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:11.253169 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005c939-6f00-4524-ac55-3ab26aeb4fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:11.253444 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005c939-6f00-4524-ac55-3ab26aeb4fb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:11.254167 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005c939-6f00-4524-ac55-3ab26aeb4fb5 None admin] RBAC: Adding query filter params (role.id=99e8fe7df8e7446680a81d4eaaa1cb68, scope.domain.id=default, user.id=53878dbb519c4b3d8858375928f1c2d4) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:11.254477 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005c939-6f00-4524-ac55-3ab26aeb4fb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:11.256429 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005c939-6f00-4524-ac55-3ab26aeb4fb5 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:11.261355 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 10/20] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:53:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:53:11.263191 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 11/21] 10.222.0.33 () {56 vars in 1322 bytes} [Sun Sep 11 14:53:11 2022] GET /identity/v3/role_assignments?user.id=53878dbb519c4b3d8858375928f1c2d4&scope.domain.id=default&role.id=99e8fe7df8e7446680a81d4eaaa1cb68 => generated 233 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:53:13.649871 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8cba2-87ab-47b3-97e6-a77fb5a89ff3 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:13.650408 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8cba2-87ab-47b3-97e6-a77fb5a89ff3 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:13.650501 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8cba2-87ab-47b3-97e6-a77fb5a89ff3 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:13.651799 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 11/22] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:13.668202 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88947a-b787-40c3-a42b-8008eaa001f2 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:13.669010 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88947a-b787-40c3-a42b-8008eaa001f2 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:13.669456 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88947a-b787-40c3-a42b-8008eaa001f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:13.694996 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-ad88947a-b787-40c3-a42b-8008eaa001f2 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:13.696448 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-ad88947a-b787-40c3-a42b-8008eaa001f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:13.792659 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 12/23] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:13.941138 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807b404-88e9-4756-81d1-1352a4961da4 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:13.941654 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807b404-88e9-4756-81d1-1352a4961da4 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:13.942074 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807b404-88e9-4756-81d1-1352a4961da4 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:13.943587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 12/24] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:13.949193 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b67d39-5a57-4c70-84d2-52707885e6d7 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:13.949704 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b67d39-5a57-4c70-84d2-52707885e6d7 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:13.950136 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b67d39-5a57-4c70-84d2-52707885e6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:13.983850 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-c5b67d39-5a57-4c70-84d2-52707885e6d7 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:13.985288 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-c5b67d39-5a57-4c70-84d2-52707885e6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:14.079629 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 13/25] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:14.109542 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb8aa8-acf6-4f08-a020-f0b3dcf913c9 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:14.110043 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb8aa8-acf6-4f08-a020-f0b3dcf913c9 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:14.110462 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb8aa8-acf6-4f08-a020-f0b3dcf913c9 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:14.112768 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 13/26] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:14.125077 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e38b7-4a17-4d35-b2b8-af0f1429aead None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:14.125592 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e38b7-4a17-4d35-b2b8-af0f1429aead None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:14.126006 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e38b7-4a17-4d35-b2b8-af0f1429aead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:14.139712 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:14.155860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-681e38b7-4a17-4d35-b2b8-af0f1429aead None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:14.157059 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-681e38b7-4a17-4d35-b2b8-af0f1429aead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:14.234239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb8aa8-acf6-4f08-a020-f0b3dcf913c9 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:14.270271 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 14/27] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:14.412440 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb8aa8-acf6-4f08-a020-f0b3dcf913c9 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:14.432924 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:14.460414 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba9431-2657-4add-a764-c46b692e1e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:14.461068 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba9431-2657-4add-a764-c46b692e1e72 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:14.461351 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba9431-2657-4add-a764-c46b692e1e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:14.461627 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba9431-2657-4add-a764-c46b692e1e72 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:14.479364 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eba9431-2657-4add-a764-c46b692e1e72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:53:14.485551 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-7eba9431-2657-4add-a764-c46b692e1e72 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:53:14.485551 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:53:14.485551 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:53:14.485551 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:53:14.485551 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:53:14.485551 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:53:14.485551 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:53:14.485551 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:53:14.485551 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:53:14.485551 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:53:14.485551 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:53:14.485551 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:53:14.485551 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:53:14.485551 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:53:14.485551 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:53:14.485551 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:53:14.486634 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:53:14.486634 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:53:14.486634 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:53:14.486634 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:53:14.486634 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:53:14.486634 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:53:14.486634 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:53:14.486634 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:53:14.486634 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:53:14.486634 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:53:14.486634 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:53:14.486634 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:53:14.486634 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:53:14.486634 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:53:14.486634 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:53:14.486634 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:53:14.487447 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:53:14.487447 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:53:14.487447 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:53:14.487447 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:53:14.487447 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:53:14.487447 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:53:14.487447 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:53:14.487447 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:53:14.487447 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:53:14.487447 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:53:14.487447 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:53:14.487447 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:53:14.487447 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 14/28] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:53:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:53:14.495378 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e38b7-4a17-4d35-b2b8-af0f1429aead None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:14.651124 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e38b7-4a17-4d35-b2b8-af0f1429aead None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:14.712021 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba9431-2657-4add-a764-c46b692e1e72 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:14.714333 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18efac27-57eb-45e8-b46e-f6a529c75bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:14.715023 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efac27-57eb-45e8-b46e-f6a529c75bd6 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:14.715309 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efac27-57eb-45e8-b46e-f6a529c75bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:14.715583 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efac27-57eb-45e8-b46e-f6a529c75bd6 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:14.716264 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18efac27-57eb-45e8-b46e-f6a529c75bd6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:14.716569 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18efac27-57eb-45e8-b46e-f6a529c75bd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:14.717980 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18efac27-57eb-45e8-b46e-f6a529c75bd6 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:14.719576 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:14.733534 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 15/29] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:53:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:14.739929 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18efac27-57eb-45e8-b46e-f6a529c75bd6 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:14.894920 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18efac27-57eb-45e8-b46e-f6a529c75bd6 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:14.910986 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba9431-2657-4add-a764-c46b692e1e72 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:14.941467 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e8f91-e7dd-44a8-9281-20390686acd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:14.942300 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e8f91-e7dd-44a8-9281-20390686acd6 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:14.942587 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e8f91-e7dd-44a8-9281-20390686acd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:14.942887 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e8f91-e7dd-44a8-9281-20390686acd6 None admin] PATH_INFO: `/v3/users/53878dbb519c4b3d8858375928f1c2d4` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:14.944351 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54e8f91-e7dd-44a8-9281-20390686acd6 None admin] RBAC: Authorizing `identity:get_user(user_id=53878dbb519c4b3d8858375928f1c2d4)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:14.946627 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54e8f91-e7dd-44a8-9281-20390686acd6 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:14.948275 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 16/30] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:53:14 2022] GET /identity/v3/users/53878dbb519c4b3d8858375928f1c2d4 => generated 247 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:53:14.949047 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e8f91-e7dd-44a8-9281-20390686acd6 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:14.950527 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec38b8-ea39-4134-bf14-69091913c414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:14.951502 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec38b8-ea39-4134-bf14-69091913c414 None admin] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:14.951990 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec38b8-ea39-4134-bf14-69091913c414 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:14.952468 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec38b8-ea39-4134-bf14-69091913c414 None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:14.953456 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec38b8-ea39-4134-bf14-69091913c414 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:14.955749 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec38b8-ea39-4134-bf14-69091913c414 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:15.026722 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:15.069701 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e8f91-e7dd-44a8-9281-20390686acd6 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:15.106544 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4602e84-cff7-4af7-8a0e-8b9358b2d072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:15.107568 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4602e84-cff7-4af7-8a0e-8b9358b2d072 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:15.108009 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4602e84-cff7-4af7-8a0e-8b9358b2d072 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:15.108419 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4602e84-cff7-4af7-8a0e-8b9358b2d072 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:15.109117 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 15/31] 10.222.0.33 () {60 vars in 1131 bytes} [Sun Sep 11 14:53:14 2022] POST /identity/v3/roles => generated 219 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:53:15.114502 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7c563-b7be-4672-abfc-e43ca52cda85 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:15.114502 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7c563-b7be-4672-abfc-e43ca52cda85 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:15.114502 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7c563-b7be-4672-abfc-e43ca52cda85 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:15.114502 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 16/32] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:15.114502 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb7c563-b7be-4672-abfc-e43ca52cda85 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:15.132650 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4602e84-cff7-4af7-8a0e-8b9358b2d072 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:15.133991 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4602e84-cff7-4af7-8a0e-8b9358b2d072 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:15.168500 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-f4602e84-cff7-4af7-8a0e-8b9358b2d072 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:53:15.168500 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:53:15.168500 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:53:15.168500 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:53:15.168500 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:53:15.168500 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:53:15.168500 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:53:15.168500 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:53:15.168500 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:53:15.168500 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:53:15.168500 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:53:15.168500 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:53:15.168500 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:53:15.168500 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:53:15.168500 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:53:15.168500 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:53:15.169445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:53:15.169445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:53:15.169445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:53:15.169445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:53:15.169445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:53:15.169445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:53:15.169445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:53:15.169445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:53:15.169445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:53:15.169445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:53:15.169445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:53:15.169445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:53:15.169445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:53:15.169445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:53:15.169445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:53:15.169445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:53:15.169445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:53:15.170262 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:53:15.170262 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:53:15.170262 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:53:15.180780 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 17/33] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:53:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:53:15.185716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee503d-b137-4865-8bb9-d11313149774 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:15.189947 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee503d-b137-4865-8bb9-d11313149774 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:15.190543 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee503d-b137-4865-8bb9-d11313149774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:15.191223 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:15.216379 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-44ee503d-b137-4865-8bb9-d11313149774 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:15.217598 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-44ee503d-b137-4865-8bb9-d11313149774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:15.312390 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 18/34] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:15.313202 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee503d-b137-4865-8bb9-d11313149774 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:15.363610 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb7c563-b7be-4672-abfc-e43ca52cda85 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:15.387596 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:15.450652 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af501f8b-db82-4440-9dd6-702fe364f9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:15.451328 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af501f8b-db82-4440-9dd6-702fe364f9e4 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:15.451609 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af501f8b-db82-4440-9dd6-702fe364f9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:15.451882 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af501f8b-db82-4440-9dd6-702fe364f9e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:15.455694 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af501f8b-db82-4440-9dd6-702fe364f9e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:15.458231 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af501f8b-db82-4440-9dd6-702fe364f9e4 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:15.459373 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 17/35] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:53:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:15.460527 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22bc73-d930-4765-9947-262a7c6e4caa None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:15.460812 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22bc73-d930-4765-9947-262a7c6e4caa None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:15.461089 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22bc73-d930-4765-9947-262a7c6e4caa None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:15.462295 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 18/36] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:15.469984 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d968e-c2ea-4909-9998-44f9934fc27c None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:15.471627 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d968e-c2ea-4909-9998-44f9934fc27c None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:15.472048 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d968e-c2ea-4909-9998-44f9934fc27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:15.494171 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-8f6d968e-c2ea-4909-9998-44f9934fc27c None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:15.496541 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-8f6d968e-c2ea-4909-9998-44f9934fc27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:15.502330 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee503d-b137-4865-8bb9-d11313149774 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:15.604740 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5bd32-0e9d-4516-a219-77b41d8d2d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:15.607508 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5bd32-0e9d-4516-a219-77b41d8d2d89 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:15.609707 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5bd32-0e9d-4516-a219-77b41d8d2d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:15.611828 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5bd32-0e9d-4516-a219-77b41d8d2d89 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:15.614194 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f5bd32-0e9d-4516-a219-77b41d8d2d89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:15.615457 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f5bd32-0e9d-4516-a219-77b41d8d2d89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:15.621316 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 19/37] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:15.621572 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f5bd32-0e9d-4516-a219-77b41d8d2d89 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:15.622360 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d968e-c2ea-4909-9998-44f9934fc27c None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:15.666079 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 19/38] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:53:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:15.672674 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5bd32-0e9d-4516-a219-77b41d8d2d89 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:15.698296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:15.777087 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d968e-c2ea-4909-9998-44f9934fc27c None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:15.820046 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5bd32-0e9d-4516-a219-77b41d8d2d89 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:15.822111 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e776f20-572b-4fe4-856d-5546d9564307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:15.822793 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e776f20-572b-4fe4-856d-5546d9564307 None admin] REQUEST_METHOD: `PUT` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:15.823108 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e776f20-572b-4fe4-856d-5546d9564307 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:15.823393 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e776f20-572b-4fe4-856d-5546d9564307 None admin] PATH_INFO: `/v3/domains/default/users/53878dbb519c4b3d8858375928f1c2d4/roles/99e8fe7df8e7446680a81d4eaaa1cb68` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:15.826316 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e776f20-572b-4fe4-856d-5546d9564307 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=53878dbb519c4b3d8858375928f1c2d4, role_id=99e8fe7df8e7446680a81d4eaaa1cb68)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:15.828029 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:15.828772 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e776f20-572b-4fe4-856d-5546d9564307 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:15.860328 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 20/39] 10.222.0.33 () {58 vars in 1361 bytes} [Sun Sep 11 14:53:15 2022] PUT /identity/v3/domains/default/users/53878dbb519c4b3d8858375928f1c2d4/roles/99e8fe7df8e7446680a81d4eaaa1cb68 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:53:15.860328 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e776f20-572b-4fe4-856d-5546d9564307 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:15.864757 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0dd8d-3bb4-4d34-bd28-9713e36ec554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:15.866621 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0dd8d-3bb4-4d34-bd28-9713e36ec554 None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:15.866621 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0dd8d-3bb4-4d34-bd28-9713e36ec554 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:15.866621 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0dd8d-3bb4-4d34-bd28-9713e36ec554 None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:15.867198 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea0dd8d-3bb4-4d34-bd28-9713e36ec554 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:15.869342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea0dd8d-3bb4-4d34-bd28-9713e36ec554 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:15.982423 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:15.992135 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e776f20-572b-4fe4-856d-5546d9564307 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:16.037107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d49bb-c16f-4864-9401-1b6e7638814b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:16.037753 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d49bb-c16f-4864-9401-1b6e7638814b None admin] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:16.038039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d49bb-c16f-4864-9401-1b6e7638814b None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:16.038318 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d49bb-c16f-4864-9401-1b6e7638814b None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:16.039045 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804d49bb-c16f-4864-9401-1b6e7638814b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:16.040553 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804d49bb-c16f-4864-9401-1b6e7638814b None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:16.062709 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 21/40] 10.222.0.33 () {60 vars in 1131 bytes} [Sun Sep 11 14:53:15 2022] POST /identity/v3/roles => generated 225 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:53:16.064811 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d980210-7da8-4cbc-88ee-afd6164f62ef None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:16.065185 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d980210-7da8-4cbc-88ee-afd6164f62ef None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:16.065506 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d980210-7da8-4cbc-88ee-afd6164f62ef None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:16.066986 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 22/41] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:16.067735 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d980210-7da8-4cbc-88ee-afd6164f62ef None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:16.137976 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 20/42] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 11 14:53:15 2022] POST /identity/v3/projects => generated 301 bytes in 466 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 14:53:16.143683 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91945e-1937-4ea1-af02-e2c68760be90 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:16.143974 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91945e-1937-4ea1-af02-e2c68760be90 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:16.144255 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91945e-1937-4ea1-af02-e2c68760be90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:16.192031 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-7b91945e-1937-4ea1-af02-e2c68760be90 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:16.193640 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7b91945e-1937-4ea1-af02-e2c68760be90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:16.198774 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d980210-7da8-4cbc-88ee-afd6164f62ef None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:16.249440 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a7ca8-77fb-4514-907f-8809635c3deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:16.250398 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a7ca8-77fb-4514-907f-8809635c3deb None admin] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:16.250845 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a7ca8-77fb-4514-907f-8809635c3deb None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:16.251323 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a7ca8-77fb-4514-907f-8809635c3deb None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:16.252383 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39a7ca8-77fb-4514-907f-8809635c3deb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:16.254530 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39a7ca8-77fb-4514-907f-8809635c3deb None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:16.276285 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 23/43] 10.222.0.33 () {60 vars in 1131 bytes} [Sun Sep 11 14:53:16 2022] POST /identity/v3/roles => generated 223 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:53:16.333378 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 21/44] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:16.616201 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a7ca8-77fb-4514-907f-8809635c3deb None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:16.633220 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:16.717099 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a7ca8-77fb-4514-907f-8809635c3deb None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:16.750472 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f9d213-50c3-4f03-b9c5-808c02d00374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:16.751120 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9d213-50c3-4f03-b9c5-808c02d00374 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:16.751857 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9d213-50c3-4f03-b9c5-808c02d00374 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:16.752130 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9d213-50c3-4f03-b9c5-808c02d00374 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:16.753245 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f9d213-50c3-4f03-b9c5-808c02d00374 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:16.754692 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f9d213-50c3-4f03-b9c5-808c02d00374 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:16.756278 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 24/45] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:53:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:16.761921 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91945e-1937-4ea1-af02-e2c68760be90 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:16.855043 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:16.868980 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91945e-1937-4ea1-af02-e2c68760be90 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:16.901650 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012bf63-a70b-4d28-8ba7-e843c28535ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:16.902257 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012bf63-a70b-4d28-8ba7-e843c28535ce None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:16.902538 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012bf63-a70b-4d28-8ba7-e843c28535ce None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:16.902832 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012bf63-a70b-4d28-8ba7-e843c28535ce None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:16.903482 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012bf63-a70b-4d28-8ba7-e843c28535ce None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:16.905068 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012bf63-a70b-4d28-8ba7-e843c28535ce None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:16.931230 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 22/46] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 11 14:53:16 2022] POST /identity/v3/projects => generated 312 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:53:18.081915 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afb00f-4330-4fee-bcfe-692b7836ac5c None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:18.082477 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afb00f-4330-4fee-bcfe-692b7836ac5c None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:18.082477 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afb00f-4330-4fee-bcfe-692b7836ac5c None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:18.083851 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 25/47] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:18.089318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23cdb7-9747-4481-a293-0b78e7219748 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:18.089736 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23cdb7-9747-4481-a293-0b78e7219748 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:18.089888 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23cdb7-9747-4481-a293-0b78e7219748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:18.108479 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-ef23cdb7-9747-4481-a293-0b78e7219748 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:18.109678 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-ef23cdb7-9747-4481-a293-0b78e7219748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:18.179331 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 23/48] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:18.447884 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26afb00f-4330-4fee-bcfe-692b7836ac5c None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:18.531186 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:18.550405 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26afb00f-4330-4fee-bcfe-692b7836ac5c None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:18.584325 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366de12-8bf0-4aba-9fd9-ce476cbe2d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:18.584957 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366de12-8bf0-4aba-9fd9-ce476cbe2d8e None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:18.585241 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366de12-8bf0-4aba-9fd9-ce476cbe2d8e None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:18.585519 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366de12-8bf0-4aba-9fd9-ce476cbe2d8e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:18.596017 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366de12-8bf0-4aba-9fd9-ce476cbe2d8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:53:18.600891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-e366de12-8bf0-4aba-9fd9-ce476cbe2d8e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:53:18.600891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:53:18.600891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:53:18.600891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:53:18.600891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:53:18.600891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:53:18.600891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:53:18.600891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:53:18.600891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:53:18.600891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:53:18.600891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:53:18.600891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:53:18.600891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:53:18.600891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:53:18.600891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:53:18.600891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:53:18.601728 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:53:18.601728 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:53:18.601728 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:53:18.601728 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:53:18.601728 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:53:18.601728 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:53:18.601728 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:53:18.601728 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:53:18.601728 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:53:18.601728 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:53:18.601728 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:53:18.601728 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:53:18.601728 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:53:18.601728 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:53:18.601728 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:53:18.601728 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:53:18.602485 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:53:18.602485 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:53:18.602485 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:53:18.602485 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:53:18.602485 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:53:18.602485 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:53:18.602485 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:53:18.602485 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:53:18.602485 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:53:18.602485 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:53:18.602485 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:53:18.602485 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:53:18.602485 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 26/49] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:53:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:53:18.607847 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef23cdb7-9747-4481-a293-0b78e7219748 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:18.679713 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:18.713211 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef23cdb7-9747-4481-a293-0b78e7219748 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:18.747946 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac25ed4-7584-4fea-aebc-6511cbe6f1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:18.748561 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac25ed4-7584-4fea-aebc-6511cbe6f1c0 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:18.748934 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac25ed4-7584-4fea-aebc-6511cbe6f1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:18.749221 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac25ed4-7584-4fea-aebc-6511cbe6f1c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:18.749888 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac25ed4-7584-4fea-aebc-6511cbe6f1c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:18.750196 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac25ed4-7584-4fea-aebc-6511cbe6f1c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:18.752292 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac25ed4-7584-4fea-aebc-6511cbe6f1c0 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:18.760361 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 24/50] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:53:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:18.766055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366de12-8bf0-4aba-9fd9-ce476cbe2d8e None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:18.866453 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366de12-8bf0-4aba-9fd9-ce476cbe2d8e None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:18.900105 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc38a9-aa8d-4390-bdf5-29e3ce7b8f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:18.900741 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc38a9-aa8d-4390-bdf5-29e3ce7b8f68 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:18.901022 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc38a9-aa8d-4390-bdf5-29e3ce7b8f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:18.901304 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc38a9-aa8d-4390-bdf5-29e3ce7b8f68 None admin] PATH_INFO: `/v3/users/53878dbb519c4b3d8858375928f1c2d4` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:18.902441 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc38a9-aa8d-4390-bdf5-29e3ce7b8f68 None admin] RBAC: Authorizing `identity:get_user(user_id=53878dbb519c4b3d8858375928f1c2d4)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:18.903949 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc38a9-aa8d-4390-bdf5-29e3ce7b8f68 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:18.905562 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 27/51] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:53:18 2022] GET /identity/v3/users/53878dbb519c4b3d8858375928f1c2d4 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:53:18.912011 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac25ed4-7584-4fea-aebc-6511cbe6f1c0 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:19.017908 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac25ed4-7584-4fea-aebc-6511cbe6f1c0 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:19.046710 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:19.052177 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b6c29-c134-4c65-8cd3-f9e9ed274482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:19.052809 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b6c29-c134-4c65-8cd3-f9e9ed274482 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:19.053092 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b6c29-c134-4c65-8cd3-f9e9ed274482 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:19.053365 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b6c29-c134-4c65-8cd3-f9e9ed274482 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:19.054471 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b6c29-c134-4c65-8cd3-f9e9ed274482 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:19.055968 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b6c29-c134-4c65-8cd3-f9e9ed274482 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:19.057818 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 25/52] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:53:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:19.063562 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc38a9-aa8d-4390-bdf5-29e3ce7b8f68 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:19.076520 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:19.089704 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba8530-fd77-4a18-a1f7-37c2a7a47aa4 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:19.090178 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba8530-fd77-4a18-a1f7-37c2a7a47aa4 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:19.090384 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba8530-fd77-4a18-a1f7-37c2a7a47aa4 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:19.091801 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 26/53] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:53:19.097450 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc749d4-279a-434c-8ffd-42311de90dec None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:19.097746 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc749d4-279a-434c-8ffd-42311de90dec None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:19.098026 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc749d4-279a-434c-8ffd-42311de90dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:19.115560 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-4bc749d4-279a-434c-8ffd-42311de90dec None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:19.116773 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-4bc749d4-279a-434c-8ffd-42311de90dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:19.167368 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc38a9-aa8d-4390-bdf5-29e3ce7b8f68 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:19.183490 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 27/54] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:53:19.201430 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd2589-a775-473e-ac62-4756a130179d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:19.202042 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd2589-a775-473e-ac62-4756a130179d None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:19.202322 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd2589-a775-473e-ac62-4756a130179d None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:19.202593 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd2589-a775-473e-ac62-4756a130179d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:19.203342 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd2589-a775-473e-ac62-4756a130179d None admin] RBAC: Adding query filter params (role.id=99e8fe7df8e7446680a81d4eaaa1cb68, scope.domain.id=default, user.id=53878dbb519c4b3d8858375928f1c2d4) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:19.203675 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd2589-a775-473e-ac62-4756a130179d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:19.205675 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd2589-a775-473e-ac62-4756a130179d None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:19.213489 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 28/55] 10.222.0.33 () {56 vars in 1322 bytes} [Sun Sep 11 14:53:19 2022] GET /identity/v3/role_assignments?user.id=53878dbb519c4b3d8858375928f1c2d4&scope.domain.id=default&role.id=99e8fe7df8e7446680a81d4eaaa1cb68 => generated 530 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:19.453328 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc749d4-279a-434c-8ffd-42311de90dec None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:19.551726 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc749d4-279a-434c-8ffd-42311de90dec None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:19.581479 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:19.587335 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa60550-91b6-4591-a267-87ca7f2f7b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:19.587970 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa60550-91b6-4591-a267-87ca7f2f7b18 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:19.588256 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa60550-91b6-4591-a267-87ca7f2f7b18 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:19.588536 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa60550-91b6-4591-a267-87ca7f2f7b18 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:19.589649 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa60550-91b6-4591-a267-87ca7f2f7b18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:19.591155 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa60550-91b6-4591-a267-87ca7f2f7b18 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:19.592730 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 28/56] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:53:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:19.598307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd2589-a775-473e-ac62-4756a130179d None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:19.671608 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:19.697296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd2589-a775-473e-ac62-4756a130179d None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:19.729618 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536f51a0-ad4e-4efd-a127-ef999e9bdc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:19.730226 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f51a0-ad4e-4efd-a127-ef999e9bdc19 None admin] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:19.730508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f51a0-ad4e-4efd-a127-ef999e9bdc19 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:19.730783 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f51a0-ad4e-4efd-a127-ef999e9bdc19 None admin] PATH_INFO: `/v3/projects` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:19.731458 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536f51a0-ad4e-4efd-a127-ef999e9bdc19 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:19.732908 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536f51a0-ad4e-4efd-a127-ef999e9bdc19 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:19.760534 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 29/57] 10.222.0.33 () {60 vars in 1140 bytes} [Sun Sep 11 14:53:19 2022] POST /identity/v3/projects => generated 298 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:53:21.893800 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120c887-c543-4157-93d8-7ba87d1ae3d7 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:21.894321 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120c887-c543-4157-93d8-7ba87d1ae3d7 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:21.894429 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120c887-c543-4157-93d8-7ba87d1ae3d7 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:21.895742 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 29/58] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:21.900966 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e404c-1608-40a3-bbe8-4c9546caebf4 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:21.901356 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e404c-1608-40a3-bbe8-4c9546caebf4 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:21.901514 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e404c-1608-40a3-bbe8-4c9546caebf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:21.921156 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-676e404c-1608-40a3-bbe8-4c9546caebf4 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:21.922367 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-676e404c-1608-40a3-bbe8-4c9546caebf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:22.092085 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 30/59] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:22.363305 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4120c887-c543-4157-93d8-7ba87d1ae3d7 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:22.463020 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4120c887-c543-4157-93d8-7ba87d1ae3d7 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:22.495601 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac3c9a0-1e93-4126-b7d5-a251de3f934c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:22.496222 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3c9a0-1e93-4126-b7d5-a251de3f934c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:22.496505 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3c9a0-1e93-4126-b7d5-a251de3f934c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:22.496777 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3c9a0-1e93-4126-b7d5-a251de3f934c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:22.497878 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac3c9a0-1e93-4126-b7d5-a251de3f934c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:22.499375 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac3c9a0-1e93-4126-b7d5-a251de3f934c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:22.500927 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 30/60] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:53:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:22.506362 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e404c-1608-40a3-bbe8-4c9546caebf4 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:22.564399 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:22.608043 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e404c-1608-40a3-bbe8-4c9546caebf4 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:22.641374 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911568a-dfa3-4a8c-a61b-6df24a2b1f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:22.642116 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911568a-dfa3-4a8c-a61b-6df24a2b1f2f None admin] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:22.642403 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911568a-dfa3-4a8c-a61b-6df24a2b1f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:22.642685 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911568a-dfa3-4a8c-a61b-6df24a2b1f2f None admin] PATH_INFO: `/v3/users` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:22.643383 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f911568a-dfa3-4a8c-a61b-6df24a2b1f2f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:22.644849 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f911568a-dfa3-4a8c-a61b-6df24a2b1f2f None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:22.670666 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 31/61] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 14:53:22 2022] POST /identity/v3/users => generated 275 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:53:26.049587 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fba353-af50-416f-b621-ea15e81185fe None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:26.050445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fba353-af50-416f-b621-ea15e81185fe None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:26.050911 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fba353-af50-416f-b621-ea15e81185fe None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:26.052650 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 31/62] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:26.065512 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f56c2-a355-4e4b-94ab-23cb9938cc11 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:26.066376 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f56c2-a355-4e4b-94ab-23cb9938cc11 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:26.066789 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f56c2-a355-4e4b-94ab-23cb9938cc11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:26.096906 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-790f56c2-a355-4e4b-94ab-23cb9938cc11 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:26.098421 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-790f56c2-a355-4e4b-94ab-23cb9938cc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:26.201405 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 32/63] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:26.294674 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2450cd-e860-4590-b4c4-0798af583122 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:26.295032 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2450cd-e860-4590-b4c4-0798af583122 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:26.295212 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2450cd-e860-4590-b4c4-0798af583122 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:26.296537 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 32/64] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:26.305146 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05198dc-35d6-4f1e-b3b1-86f0d637c0ff None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:26.305433 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05198dc-35d6-4f1e-b3b1-86f0d637c0ff None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:26.305712 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05198dc-35d6-4f1e-b3b1-86f0d637c0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:26.331955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-d05198dc-35d6-4f1e-b3b1-86f0d637c0ff None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:26.333185 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-d05198dc-35d6-4f1e-b3b1-86f0d637c0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:26.372767 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62125ec3-83e5-4a66-be34-de690cec412d None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:26.373065 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62125ec3-83e5-4a66-be34-de690cec412d None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:26.373332 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62125ec3-83e5-4a66-be34-de690cec412d None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:26.374731 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 33/65] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:26.382782 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:26.391001 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9496bb8-1042-4d4e-ad58-71302d772109 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:26.391298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9496bb8-1042-4d4e-ad58-71302d772109 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:26.391569 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9496bb8-1042-4d4e-ad58-71302d772109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:26.415739 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-b9496bb8-1042-4d4e-ad58-71302d772109 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:26.417219 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-b9496bb8-1042-4d4e-ad58-71302d772109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:26.455209 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 33/66] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:26.501276 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c01bc5-c7c9-4f8e-b8d6-7051fbd66907 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:26.501582 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c01bc5-c7c9-4f8e-b8d6-7051fbd66907 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:26.501854 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c01bc5-c7c9-4f8e-b8d6-7051fbd66907 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:26.503423 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 34/67] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:53:26.507728 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c01bc5-c7c9-4f8e-b8d6-7051fbd66907 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:26.520297 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:26.524908 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 34/68] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:53:26.526095 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959d2fe-9a8d-4618-b056-c91a35fcabf8 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:26.526383 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959d2fe-9a8d-4618-b056-c91a35fcabf8 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:26.526664 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959d2fe-9a8d-4618-b056-c91a35fcabf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:26.546733 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-2959d2fe-9a8d-4618-b056-c91a35fcabf8 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:26.548471 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-2959d2fe-9a8d-4618-b056-c91a35fcabf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:26.622550 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 35/69] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:26.623939 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba4880-3e41-4981-961d-94e0780f3729 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:26.624233 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba4880-3e41-4981-961d-94e0780f3729 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:26.624509 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba4880-3e41-4981-961d-94e0780f3729 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:26.625897 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 36/70] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:26.631371 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc4a82-903a-489a-a9b8-92597e558059 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:26.631658 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc4a82-903a-489a-a9b8-92597e558059 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:26.631935 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc4a82-903a-489a-a9b8-92597e558059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:26.649287 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c01bc5-c7c9-4f8e-b8d6-7051fbd66907 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:26.650366 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-7ffc4a82-903a-489a-a9b8-92597e558059 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:26.651593 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7ffc4a82-903a-489a-a9b8-92597e558059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:26.693821 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79510188-c33e-40ac-a44f-5b7da7cffbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:26.694471 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79510188-c33e-40ac-a44f-5b7da7cffbc6 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:26.694761 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79510188-c33e-40ac-a44f-5b7da7cffbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:26.695089 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79510188-c33e-40ac-a44f-5b7da7cffbc6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:26.708429 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79510188-c33e-40ac-a44f-5b7da7cffbc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:53:26.714618 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-79510188-c33e-40ac-a44f-5b7da7cffbc6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:53:26.714618 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:53:26.714618 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:53:26.714618 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:53:26.714618 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:53:26.714618 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:53:26.714618 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:53:26.714618 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:53:26.714618 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:53:26.714618 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:53:26.714618 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:53:26.714618 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:53:26.714618 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:53:26.714618 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:53:26.714618 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:53:26.714618 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:53:26.716002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:53:26.716002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:53:26.716002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:53:26.716002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:53:26.716002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:53:26.716002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:53:26.716002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:53:26.716002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:53:26.716002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:53:26.716002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:53:26.716002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:53:26.716002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:53:26.716002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:53:26.716002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:53:26.716002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:53:26.716002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:53:26.716780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:53:26.716780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:53:26.716780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:53:26.716780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:53:26.716780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:53:26.716780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:53:26.716780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:53:26.716780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:53:26.716780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:53:26.716780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:53:26.716780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:53:26.716780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:53:26.716780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 35/71] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:53:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:53:26.722607 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79510188-c33e-40ac-a44f-5b7da7cffbc6 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:26.737338 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 37/72] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:53:26.755390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:26.830754 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79510188-c33e-40ac-a44f-5b7da7cffbc6 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:26.833591 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc4a82-903a-489a-a9b8-92597e558059 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:26.881364 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62485eaa-5a45-46f8-a756-f42095b2fc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:26.881983 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62485eaa-5a45-46f8-a756-f42095b2fc2b None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:26.882308 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62485eaa-5a45-46f8-a756-f42095b2fc2b None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:26.882600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62485eaa-5a45-46f8-a756-f42095b2fc2b None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:26.883448 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62485eaa-5a45-46f8-a756-f42095b2fc2b None admin] RBAC: Adding query filter params (name=member) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:26.883758 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62485eaa-5a45-46f8-a756-f42095b2fc2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:26.885170 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62485eaa-5a45-46f8-a756-f42095b2fc2b None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:26.898142 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 36/73] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 14:53:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:53:26.898989 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62485eaa-5a45-46f8-a756-f42095b2fc2b None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:26.910037 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:26.963980 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc4a82-903a-489a-a9b8-92597e558059 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:27.005958 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcff41-4d0a-4f64-a89b-f48dd4bcd679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:27.006602 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcff41-4d0a-4f64-a89b-f48dd4bcd679 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:27.006921 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcff41-4d0a-4f64-a89b-f48dd4bcd679 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:27.007200 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcff41-4d0a-4f64-a89b-f48dd4bcd679 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:27.013034 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:27.018533 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bcff41-4d0a-4f64-a89b-f48dd4bcd679 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:53:27.021559 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62485eaa-5a45-46f8-a756-f42095b2fc2b None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:27.023539 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-a4bcff41-4d0a-4f64-a89b-f48dd4bcd679 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:53:27.023539 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:53:27.023539 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:53:27.023539 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:53:27.023539 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:53:27.023539 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:53:27.023539 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:53:27.023539 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:53:27.023539 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:53:27.023539 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:53:27.023539 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:53:27.023539 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:53:27.023539 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:53:27.023539 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:53:27.023539 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:53:27.023539 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:53:27.024379 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:53:27.024379 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:53:27.024379 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:53:27.024379 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:53:27.024379 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:53:27.024379 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:53:27.024379 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:53:27.024379 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:53:27.024379 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:53:27.024379 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:53:27.024379 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:53:27.024379 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:53:27.024379 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:53:27.024379 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:53:27.024379 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:53:27.024379 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:53:27.025165 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:53:27.025165 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:53:27.025165 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:53:27.025165 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:53:27.025165 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:53:27.025165 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:53:27.025165 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:53:27.025165 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:53:27.025165 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:53:27.025165 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:53:27.025165 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:53:27.025165 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:53:27.025165 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 38/74] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:53:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:53:27.027318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcff41-4d0a-4f64-a89b-f48dd4bcd679 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:27.063302 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f76583e-1b05-46c3-a572-42f3ea40cd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:27.064519 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76583e-1b05-46c3-a572-42f3ea40cd79 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:27.064902 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76583e-1b05-46c3-a572-42f3ea40cd79 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:27.065186 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76583e-1b05-46c3-a572-42f3ea40cd79 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:27.075350 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f76583e-1b05-46c3-a572-42f3ea40cd79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:53:27.080298 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-7f76583e-1b05-46c3-a572-42f3ea40cd79 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:53:27.080298 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:53:27.080298 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:53:27.080298 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:53:27.080298 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:53:27.080298 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:53:27.080298 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:53:27.080298 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:53:27.080298 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:53:27.080298 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:53:27.080298 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:53:27.080298 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:53:27.080298 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:53:27.080298 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:53:27.080298 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:53:27.080298 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:53:27.081239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:53:27.081239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:53:27.081239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:53:27.081239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:53:27.081239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:53:27.081239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:53:27.081239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:53:27.081239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:53:27.081239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:53:27.081239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:53:27.081239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:53:27.081239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:53:27.081239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:53:27.081239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:53:27.081239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:53:27.081239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:53:27.082022 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:53:27.082022 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:53:27.082022 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:53:27.082022 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:53:27.082022 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:53:27.082022 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:53:27.082022 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:53:27.082022 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:53:27.082022 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:53:27.082022 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:53:27.082022 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:53:27.082022 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:53:27.086237 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 37/75] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:53:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:53:27.086942 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f76583e-1b05-46c3-a572-42f3ea40cd79 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:27.130042 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcff41-4d0a-4f64-a89b-f48dd4bcd679 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:27.146289 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:27.164056 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4998c4-ece2-4d08-b213-78c6f8d81959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:27.164689 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4998c4-ece2-4d08-b213-78c6f8d81959 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:27.164973 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4998c4-ece2-4d08-b213-78c6f8d81959 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:27.165251 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4998c4-ece2-4d08-b213-78c6f8d81959 None admin] PATH_INFO: `/v3/users/4585768734fc4332a61c845383c1482c` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:27.178846 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4998c4-ece2-4d08-b213-78c6f8d81959 None admin] RBAC: Authorizing `identity:get_user(user_id=4585768734fc4332a61c845383c1482c)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:27.180324 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4998c4-ece2-4d08-b213-78c6f8d81959 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:27.181938 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 39/76] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:53:27 2022] GET /identity/v3/users/4585768734fc4332a61c845383c1482c => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:27.181938 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4998c4-ece2-4d08-b213-78c6f8d81959 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:27.186876 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f76583e-1b05-46c3-a572-42f3ea40cd79 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:27.221556 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c6223-d283-4c23-a9ec-0af5f21b611e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:27.222184 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c6223-d283-4c23-a9ec-0af5f21b611e None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:27.222474 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c6223-d283-4c23-a9ec-0af5f21b611e None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:27.222754 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c6223-d283-4c23-a9ec-0af5f21b611e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:27.232793 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c6223-d283-4c23-a9ec-0af5f21b611e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:53:27.238188 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-441c6223-d283-4c23-a9ec-0af5f21b611e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:53:27.238188 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:53:27.238188 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:53:27.238188 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:53:27.238188 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:53:27.238188 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:53:27.238188 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:53:27.238188 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:53:27.238188 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:53:27.238188 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:53:27.238188 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:53:27.238188 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:53:27.238188 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:53:27.238188 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:53:27.238188 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:53:27.238188 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:53:27.239225 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:53:27.239225 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:53:27.239225 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:53:27.239225 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:53:27.239225 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:53:27.239225 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:53:27.239225 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:53:27.239225 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:53:27.239225 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:53:27.239225 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:53:27.239225 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:53:27.239225 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:53:27.239225 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:53:27.239225 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:53:27.239225 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:53:27.239225 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:53:27.240005 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:53:27.240005 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:53:27.240005 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:53:27.240005 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:53:27.240005 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:53:27.240005 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:53:27.240005 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:53:27.240005 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:53:27.240005 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:53:27.240005 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:53:27.240005 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:53:27.240005 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:53:27.240005 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 38/77] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 14:53:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:53:27.240746 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c6223-d283-4c23-a9ec-0af5f21b611e None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:27.255840 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:27.286713 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4998c4-ece2-4d08-b213-78c6f8d81959 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:27.320708 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a56b4bc-2cdb-4372-a979-b4afbf153d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:27.321319 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56b4bc-2cdb-4372-a979-b4afbf153d94 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:27.321603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56b4bc-2cdb-4372-a979-b4afbf153d94 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:27.321878 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56b4bc-2cdb-4372-a979-b4afbf153d94 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:27.322538 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a56b4bc-2cdb-4372-a979-b4afbf153d94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:27.322869 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a56b4bc-2cdb-4372-a979-b4afbf153d94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:27.324287 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a56b4bc-2cdb-4372-a979-b4afbf153d94 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:27.332733 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 40/78] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:53:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:27.333462 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a56b4bc-2cdb-4372-a979-b4afbf153d94 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:27.340492 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c6223-d283-4c23-a9ec-0af5f21b611e None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:27.368697 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:27.374695 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8f6d8-4635-4c4b-8552-487b344ca406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:27.375354 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8f6d8-4635-4c4b-8552-487b344ca406 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:27.375640 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8f6d8-4635-4c4b-8552-487b344ca406 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:27.376440 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8f6d8-4635-4c4b-8552-487b344ca406 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:27.377586 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de8f6d8-4635-4c4b-8552-487b344ca406 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:27.379097 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de8f6d8-4635-4c4b-8552-487b344ca406 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:27.380677 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 39/79] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:53:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:27.381440 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8f6d8-4635-4c4b-8552-487b344ca406 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:27.436747 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a56b4bc-2cdb-4372-a979-b4afbf153d94 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:27.464587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:27.471744 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a86b2f-1cbe-4a37-abf4-5576ff78ac9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:27.472359 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a86b2f-1cbe-4a37-abf4-5576ff78ac9b None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:27.472649 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a86b2f-1cbe-4a37-abf4-5576ff78ac9b None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:27.472927 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a86b2f-1cbe-4a37-abf4-5576ff78ac9b None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:27.473597 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a86b2f-1cbe-4a37-abf4-5576ff78ac9b None admin] RBAC: Adding query filter params (name=member) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:27.473908 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a86b2f-1cbe-4a37-abf4-5576ff78ac9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:27.475359 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a86b2f-1cbe-4a37-abf4-5576ff78ac9b None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:27.483372 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 41/80] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 14:53:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:27.483771 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8f6d8-4635-4c4b-8552-487b344ca406 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:27.487682 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a86b2f-1cbe-4a37-abf4-5576ff78ac9b None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:27.518336 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e3e71-e8df-495d-af97-b6edc7e0fb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:27.518982 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e3e71-e8df-495d-af97-b6edc7e0fb69 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:27.519267 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e3e71-e8df-495d-af97-b6edc7e0fb69 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:27.519542 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e3e71-e8df-495d-af97-b6edc7e0fb69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:27.588079 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:27.591055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e3e71-e8df-495d-af97-b6edc7e0fb69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:27.592180 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a86b2f-1cbe-4a37-abf4-5576ff78ac9b None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:27.592525 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e3e71-e8df-495d-af97-b6edc7e0fb69 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:27.629770 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe9666-f3ae-440b-b795-0122862d1e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:27.630393 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe9666-f3ae-440b-b795-0122862d1e33 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:27.630677 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe9666-f3ae-440b-b795-0122862d1e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:27.630989 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe9666-f3ae-440b-b795-0122862d1e33 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:27.631663 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbe9666-f3ae-440b-b795-0122862d1e33 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:27.631975 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbe9666-f3ae-440b-b795-0122862d1e33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:27.633376 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbe9666-f3ae-440b-b795-0122862d1e33 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:27.641253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 42/81] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:53:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:27.642614 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe9666-f3ae-440b-b795-0122862d1e33 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:27.683424 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:27.703099 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 40/82] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:53:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:53:27.703988 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e3e71-e8df-495d-af97-b6edc7e0fb69 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:27.746457 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe9666-f3ae-440b-b795-0122862d1e33 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:27.780048 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8fb523-518c-490e-a7cc-200511db3fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:27.780680 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8fb523-518c-490e-a7cc-200511db3fa1 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:27.780963 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8fb523-518c-490e-a7cc-200511db3fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:27.781236 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8fb523-518c-490e-a7cc-200511db3fa1 None admin] PATH_INFO: `/v3/users/53878dbb519c4b3d8858375928f1c2d4` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:27.782420 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8fb523-518c-490e-a7cc-200511db3fa1 None admin] RBAC: Authorizing `identity:get_user(user_id=53878dbb519c4b3d8858375928f1c2d4)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:27.783908 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8fb523-518c-490e-a7cc-200511db3fa1 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:27.785658 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 43/83] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:53:27 2022] GET /identity/v3/users/53878dbb519c4b3d8858375928f1c2d4 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:27.786304 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8fb523-518c-490e-a7cc-200511db3fa1 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:27.810385 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e3e71-e8df-495d-af97-b6edc7e0fb69 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:27.815087 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:27.845866 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b5588-a28e-45c1-a023-7b54cc9a4caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:27.846403 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b5588-a28e-45c1-a023-7b54cc9a4caa None admin] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:27.846685 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b5588-a28e-45c1-a023-7b54cc9a4caa None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:27.846987 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b5588-a28e-45c1-a023-7b54cc9a4caa None admin] PATH_INFO: `/v3/users` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:27.847640 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727b5588-a28e-45c1-a023-7b54cc9a4caa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:27.849083 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727b5588-a28e-45c1-a023-7b54cc9a4caa None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:27.869585 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 41/84] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 14:53:27 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:53:27.870475 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b5588-a28e-45c1-a023-7b54cc9a4caa None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:27.894261 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8fb523-518c-490e-a7cc-200511db3fa1 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:27.919176 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:27.939577 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ddfbb5-d50c-4e6a-aa42-33908cd7d61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:27.940240 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ddfbb5-d50c-4e6a-aa42-33908cd7d61e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:27.940517 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ddfbb5-d50c-4e6a-aa42-33908cd7d61e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:27.940789 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ddfbb5-d50c-4e6a-aa42-33908cd7d61e None admin] PATH_INFO: `/v3/users/4585768734fc4332a61c845383c1482c` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:27.942106 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ddfbb5-d50c-4e6a-aa42-33908cd7d61e None admin] RBAC: Authorizing `identity:get_user(user_id=4585768734fc4332a61c845383c1482c)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:27.943698 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ddfbb5-d50c-4e6a-aa42-33908cd7d61e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:27.945559 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 44/85] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:53:27 2022] GET /identity/v3/users/4585768734fc4332a61c845383c1482c => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:27.951037 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ddfbb5-d50c-4e6a-aa42-33908cd7d61e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:28.002180 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b5588-a28e-45c1-a023-7b54cc9a4caa None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:28.047008 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e775dc-734d-4fae-9f68-769d9c788e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:28.047721 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e775dc-734d-4fae-9f68-769d9c788e0b None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:28.048058 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e775dc-734d-4fae-9f68-769d9c788e0b None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:28.048393 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e775dc-734d-4fae-9f68-769d9c788e0b None admin] PATH_INFO: `/v3/users/4585768734fc4332a61c845383c1482c` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:28.049577 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e775dc-734d-4fae-9f68-769d9c788e0b None admin] RBAC: Authorizing `identity:get_user(user_id=4585768734fc4332a61c845383c1482c)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:28.051658 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e775dc-734d-4fae-9f68-769d9c788e0b None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:28.053430 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 42/86] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:53:27 2022] GET /identity/v3/users/4585768734fc4332a61c845383c1482c => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:28.055781 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e775dc-734d-4fae-9f68-769d9c788e0b None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:28.065881 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:28.073382 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ddfbb5-d50c-4e6a-aa42-33908cd7d61e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:28.121890 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3182be0a-48a7-4966-b5f9-24e542c541f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:28.122911 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182be0a-48a7-4966-b5f9-24e542c541f7 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:28.123595 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182be0a-48a7-4966-b5f9-24e542c541f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:28.123595 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182be0a-48a7-4966-b5f9-24e542c541f7 None admin] PATH_INFO: `/v3/projects/3b402af2708b41df849963f81e65797e` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:28.124508 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3182be0a-48a7-4966-b5f9-24e542c541f7 None admin] RBAC: Authorizing `identity:get_project(project_id=3b402af2708b41df849963f81e65797e)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:28.126494 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3182be0a-48a7-4966-b5f9-24e542c541f7 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:28.128198 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 45/87] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:53:27 2022] GET /identity/v3/projects/3b402af2708b41df849963f81e65797e => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:28.128975 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3182be0a-48a7-4966-b5f9-24e542c541f7 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:28.189481 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e775dc-734d-4fae-9f68-769d9c788e0b None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:28.198339 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:28.234326 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c54f0-fb11-4770-8efb-4462360b2c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:28.235147 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c54f0-fb11-4770-8efb-4462360b2c23 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:28.235587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c54f0-fb11-4770-8efb-4462360b2c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:28.236312 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c54f0-fb11-4770-8efb-4462360b2c23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:28.257647 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3182be0a-48a7-4966-b5f9-24e542c541f7 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:28.297562 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda387d-8634-4524-9cbc-d2ae571b1594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:28.298247 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda387d-8634-4524-9cbc-d2ae571b1594 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:28.298540 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda387d-8634-4524-9cbc-d2ae571b1594 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:28.298842 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda387d-8634-4524-9cbc-d2ae571b1594 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:28.322233 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c54f0-fb11-4770-8efb-4462360b2c23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:28.323778 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c54f0-fb11-4770-8efb-4462360b2c23 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:28.344020 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:28.389867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda387d-8634-4524-9cbc-d2ae571b1594 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:28.391342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda387d-8634-4524-9cbc-d2ae571b1594 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:28.441771 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 43/88] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:53:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:53:28.447350 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c54f0-fb11-4770-8efb-4462360b2c23 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:28.456610 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:28.497827 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 46/89] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:53:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:53:28.498520 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda387d-8634-4524-9cbc-d2ae571b1594 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:28.563871 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c54f0-fb11-4770-8efb-4462360b2c23 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:28.565806 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:28.606660 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597009e-c0ed-449b-8564-54c8740acc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:28.608420 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597009e-c0ed-449b-8564-54c8740acc66 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:28.608904 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597009e-c0ed-449b-8564-54c8740acc66 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:28.609371 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597009e-c0ed-449b-8564-54c8740acc66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:28.610097 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda387d-8634-4524-9cbc-d2ae571b1594 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:28.662348 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0862a3b5-1644-4fbd-9829-af39697670f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:28.663262 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862a3b5-1644-4fbd-9829-af39697670f4 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:28.663262 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862a3b5-1644-4fbd-9829-af39697670f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:28.663527 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862a3b5-1644-4fbd-9829-af39697670f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:28.664180 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0862a3b5-1644-4fbd-9829-af39697670f4 None admin] RBAC: Adding query filter params (role.id=c9da950c75da4939b8bd649c11015650, scope.project.id=3b402af2708b41df849963f81e65797e, user.id=4585768734fc4332a61c845383c1482c) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:28.664488 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0862a3b5-1644-4fbd-9829-af39697670f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:28.666205 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0862a3b5-1644-4fbd-9829-af39697670f4 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:28.674855 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 47/90] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:53:28 2022] GET /identity/v3/role_assignments?user.id=4585768734fc4332a61c845383c1482c&scope.project.id=3b402af2708b41df849963f81e65797e&role.id=c9da950c75da4939b8bd649c11015650 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:28.674855 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0862a3b5-1644-4fbd-9829-af39697670f4 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:28.730706 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7597009e-c0ed-449b-8564-54c8740acc66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:28.732275 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7597009e-c0ed-449b-8564-54c8740acc66 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:28.753581 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:28.866261 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0862a3b5-1644-4fbd-9829-af39697670f4 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:28.923577 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:28.932954 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 44/91] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:53:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:53:28.933658 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597009e-c0ed-449b-8564-54c8740acc66 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:28.943189 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bf6ad-0d8c-4c27-ba3d-3c015a2796d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:28.943891 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bf6ad-0d8c-4c27-ba3d-3c015a2796d0 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:28.944177 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bf6ad-0d8c-4c27-ba3d-3c015a2796d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:28.944456 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bf6ad-0d8c-4c27-ba3d-3c015a2796d0 None admin] PATH_INFO: `/v3/projects/3b402af2708b41df849963f81e65797e` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:28.951597 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561bf6ad-0d8c-4c27-ba3d-3c015a2796d0 None admin] RBAC: Authorizing `identity:get_project(project_id=3b402af2708b41df849963f81e65797e)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:28.953111 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561bf6ad-0d8c-4c27-ba3d-3c015a2796d0 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:28.954972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 48/92] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:53:28 2022] GET /identity/v3/projects/3b402af2708b41df849963f81e65797e => generated 298 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:28.955851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bf6ad-0d8c-4c27-ba3d-3c015a2796d0 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:29.105373 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597009e-c0ed-449b-8564-54c8740acc66 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:29.114038 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:29.159931 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bf6ad-0d8c-4c27-ba3d-3c015a2796d0 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:29.163758 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6676d8e8-c64b-4d06-9bc9-71fec3048c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:29.164716 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6676d8e8-c64b-4d06-9bc9-71fec3048c3c None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:29.165847 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6676d8e8-c64b-4d06-9bc9-71fec3048c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:29.166340 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6676d8e8-c64b-4d06-9bc9-71fec3048c3c None admin] PATH_INFO: `/v3/projects/3dc3bfe113c841f098a02b2b1ec28edc` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:29.175676 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6676d8e8-c64b-4d06-9bc9-71fec3048c3c None admin] RBAC: Authorizing `identity:get_project(project_id=3dc3bfe113c841f098a02b2b1ec28edc)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:29.177116 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6676d8e8-c64b-4d06-9bc9-71fec3048c3c None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:29.179161 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 45/93] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:53:28 2022] GET /identity/v3/projects/3dc3bfe113c841f098a02b2b1ec28edc => generated 312 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:29.179954 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6676d8e8-c64b-4d06-9bc9-71fec3048c3c None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:29.203081 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95963d-df62-4a67-8184-dfd6a5bd4b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:29.204271 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95963d-df62-4a67-8184-dfd6a5bd4b5a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:29.204790 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95963d-df62-4a67-8184-dfd6a5bd4b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:29.205312 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95963d-df62-4a67-8184-dfd6a5bd4b5a None admin] PATH_INFO: `/v3/projects/3b402af2708b41df849963f81e65797e` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:29.206765 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a95963d-df62-4a67-8184-dfd6a5bd4b5a None admin] RBAC: Authorizing `identity:get_project(project_id=3b402af2708b41df849963f81e65797e)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:29.208554 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a95963d-df62-4a67-8184-dfd6a5bd4b5a None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:29.210702 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 49/94] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:53:28 2022] GET /identity/v3/projects/3b402af2708b41df849963f81e65797e => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:29.211749 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95963d-df62-4a67-8184-dfd6a5bd4b5a None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:29.287850 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:29.328248 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95963d-df62-4a67-8184-dfd6a5bd4b5a None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:29.365538 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237864b-9e86-4c0e-a50c-cbc20f9f0368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:29.366610 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237864b-9e86-4c0e-a50c-cbc20f9f0368 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:29.367143 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237864b-9e86-4c0e-a50c-cbc20f9f0368 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:29.367614 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237864b-9e86-4c0e-a50c-cbc20f9f0368 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:29.368536 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d237864b-9e86-4c0e-a50c-cbc20f9f0368 None admin] RBAC: Adding query filter params (role.id=c9da950c75da4939b8bd649c11015650, scope.project.id=3dc3bfe113c841f098a02b2b1ec28edc, user.id=4585768734fc4332a61c845383c1482c) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:29.369103 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d237864b-9e86-4c0e-a50c-cbc20f9f0368 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:29.370774 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d237864b-9e86-4c0e-a50c-cbc20f9f0368 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:29.377335 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 50/95] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:53:29 2022] GET /identity/v3/role_assignments?user.id=4585768734fc4332a61c845383c1482c&scope.project.id=3dc3bfe113c841f098a02b2b1ec28edc&role.id=c9da950c75da4939b8bd649c11015650 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:29.378389 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237864b-9e86-4c0e-a50c-cbc20f9f0368 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:29.379848 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6676d8e8-c64b-4d06-9bc9-71fec3048c3c None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:29.455287 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e20f22-d661-44bd-b964-19462a9407f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:29.455836 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e20f22-d661-44bd-b964-19462a9407f7 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:29.456119 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e20f22-d661-44bd-b964-19462a9407f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:29.456392 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e20f22-d661-44bd-b964-19462a9407f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:29.457243 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e20f22-d661-44bd-b964-19462a9407f7 None admin] RBAC: Adding query filter params (role.id=99e8fe7df8e7446680a81d4eaaa1cb68, scope.project.id=3b402af2708b41df849963f81e65797e, user.id=53878dbb519c4b3d8858375928f1c2d4) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:29.457562 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e20f22-d661-44bd-b964-19462a9407f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:29.459159 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e20f22-d661-44bd-b964-19462a9407f7 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:29.465454 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 46/96] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:53:29 2022] GET /identity/v3/role_assignments?user.id=53878dbb519c4b3d8858375928f1c2d4&scope.project.id=3b402af2708b41df849963f81e65797e&role.id=99e8fe7df8e7446680a81d4eaaa1cb68 => generated 259 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:29.513125 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237864b-9e86-4c0e-a50c-cbc20f9f0368 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:29.555296 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:29.565121 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d3b0e4-3c13-46e2-a70c-552dbaf5e5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:29.565899 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3b0e4-3c13-46e2-a70c-552dbaf5e5fb None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:29.566460 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3b0e4-3c13-46e2-a70c-552dbaf5e5fb None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:29.566866 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3b0e4-3c13-46e2-a70c-552dbaf5e5fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:29.567711 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d3b0e4-3c13-46e2-a70c-552dbaf5e5fb None admin] RBAC: Adding query filter params (role.id=d1818eb7f2bd467fbf75bad5cc8e0aea, scope.project.id=3b402af2708b41df849963f81e65797e, user.id=4585768734fc4332a61c845383c1482c) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:29.568023 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d3b0e4-3c13-46e2-a70c-552dbaf5e5fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:29.569551 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d3b0e4-3c13-46e2-a70c-552dbaf5e5fb None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:29.579260 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 51/97] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:53:29 2022] GET /identity/v3/role_assignments?user.id=4585768734fc4332a61c845383c1482c&scope.project.id=3b402af2708b41df849963f81e65797e&role.id=d1818eb7f2bd467fbf75bad5cc8e0aea => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:30.599444 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63416b-aa01-4b23-8672-72ff2c0526c5 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:30.600498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63416b-aa01-4b23-8672-72ff2c0526c5 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:30.600982 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63416b-aa01-4b23-8672-72ff2c0526c5 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:30.602784 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 47/98] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:30.607910 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f504243-1970-4229-bc61-b36bcffa72b8 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:30.608721 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f504243-1970-4229-bc61-b36bcffa72b8 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:30.609226 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f504243-1970-4229-bc61-b36bcffa72b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:30.632122 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-7f504243-1970-4229-bc61-b36bcffa72b8 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:30.633662 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7f504243-1970-4229-bc61-b36bcffa72b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:30.729022 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 52/99] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:30.774694 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6133a4-6e03-48db-8252-aff687717452 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:30.775362 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6133a4-6e03-48db-8252-aff687717452 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:30.775838 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6133a4-6e03-48db-8252-aff687717452 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:30.777377 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 48/100] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:30.783068 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5e77d-080d-439a-b56d-37cb3cc393df None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:30.783619 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5e77d-080d-439a-b56d-37cb3cc393df None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:30.784109 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5e77d-080d-439a-b56d-37cb3cc393df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:30.803295 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-34a5e77d-080d-439a-b56d-37cb3cc393df None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:30.804806 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-34a5e77d-080d-439a-b56d-37cb3cc393df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:30.880154 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 53/101] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:31.208401 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6133a4-6e03-48db-8252-aff687717452 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:31.252124 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:31.341983 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6133a4-6e03-48db-8252-aff687717452 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:31.376942 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32669a87-4566-45de-a28a-927ab922fa42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:31.377588 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32669a87-4566-45de-a28a-927ab922fa42 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:31.377879 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32669a87-4566-45de-a28a-927ab922fa42 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:31.378176 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32669a87-4566-45de-a28a-927ab922fa42 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:31.379368 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32669a87-4566-45de-a28a-927ab922fa42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:31.380862 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32669a87-4566-45de-a28a-927ab922fa42 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:31.382527 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 49/102] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:53:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:31.389400 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5e77d-080d-439a-b56d-37cb3cc393df None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:31.456534 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:31.465995 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32669a87-4566-45de-a28a-927ab922fa42 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:31.545168 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5e77d-080d-439a-b56d-37cb3cc393df None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:31.593039 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe87e9-4a22-4417-aa41-bf82d2ac5330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:31.593778 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe87e9-4a22-4417-aa41-bf82d2ac5330 None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:31.594072 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe87e9-4a22-4417-aa41-bf82d2ac5330 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:31.594351 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe87e9-4a22-4417-aa41-bf82d2ac5330 None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:31.595188 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe87e9-4a22-4417-aa41-bf82d2ac5330 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:31.597175 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe87e9-4a22-4417-aa41-bf82d2ac5330 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:31.626718 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32669a87-4566-45de-a28a-927ab922fa42 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:31.630475 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 54/103] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 11 14:53:31 2022] POST /identity/v3/projects => generated 302 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:53:31.632301 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622f16a-6289-41d5-b5f3-9ef7c6a01811 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:31.633645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622f16a-6289-41d5-b5f3-9ef7c6a01811 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:31.634128 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622f16a-6289-41d5-b5f3-9ef7c6a01811 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:31.638610 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 55/104] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:31.646873 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:31.648761 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f85487-c90d-46fe-a8db-3d4571299b7d None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:31.651355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f85487-c90d-46fe-a8db-3d4571299b7d None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:31.651900 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f85487-c90d-46fe-a8db-3d4571299b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:31.701842 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-60f85487-c90d-46fe-a8db-3d4571299b7d None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:31.706967 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-60f85487-c90d-46fe-a8db-3d4571299b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:31.730706 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a5a7d-0fa6-4904-88a1-50fb639ed88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:31.731810 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a5a7d-0fa6-4904-88a1-50fb639ed88c None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:31.732289 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a5a7d-0fa6-4904-88a1-50fb639ed88c None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:31.732761 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a5a7d-0fa6-4904-88a1-50fb639ed88c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:31.781283 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a5a7d-0fa6-4904-88a1-50fb639ed88c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:53:31.789509 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-582a5a7d-0fa6-4904-88a1-50fb639ed88c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:53:31.789509 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:53:31.789509 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:53:31.789509 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:53:31.789509 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:53:31.789509 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:53:31.789509 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:53:31.789509 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:53:31.789509 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:53:31.789509 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:53:31.789509 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:53:31.789509 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:53:31.789509 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:53:31.789509 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:53:31.789509 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:53:31.789509 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:53:31.790342 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:53:31.790342 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:53:31.790342 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:53:31.790342 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:53:31.790342 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:53:31.790342 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:53:31.790342 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:53:31.790342 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:53:31.790342 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:53:31.790342 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:53:31.790342 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:53:31.790342 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:53:31.790342 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:53:31.790342 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:53:31.790342 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:53:31.790342 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:53:31.791533 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:53:31.791533 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:53:31.791533 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:53:31.791533 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:53:31.791533 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:53:31.791533 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:53:31.791533 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:53:31.791533 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:53:31.791533 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:53:31.791533 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:53:31.791533 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:53:31.791533 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:53:31.794644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 50/105] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:53:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:53:31.801751 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a5a7d-0fa6-4904-88a1-50fb639ed88c None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:31.888107 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 56/106] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:31.968566 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a5a7d-0fa6-4904-88a1-50fb639ed88c None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:32.014260 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebb0aa-dbdd-43d6-b992-3b257fff207a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:32.014917 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebb0aa-dbdd-43d6-b992-3b257fff207a None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:32.015301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebb0aa-dbdd-43d6-b992-3b257fff207a None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:32.015583 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebb0aa-dbdd-43d6-b992-3b257fff207a None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:32.016256 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acebb0aa-dbdd-43d6-b992-3b257fff207a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:32.017077 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acebb0aa-dbdd-43d6-b992-3b257fff207a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:32.018694 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acebb0aa-dbdd-43d6-b992-3b257fff207a None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:32.028349 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 51/107] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 14:53:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:32.035017 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f85487-c90d-46fe-a8db-3d4571299b7d None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:32.075976 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:32.174013 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f85487-c90d-46fe-a8db-3d4571299b7d None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:32.216870 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebb0aa-dbdd-43d6-b992-3b257fff207a None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:32.223150 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c885c40-209b-44bc-95d5-46ce57054ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:32.224116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c885c40-209b-44bc-95d5-46ce57054ebd None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:32.224608 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c885c40-209b-44bc-95d5-46ce57054ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:32.225089 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c885c40-209b-44bc-95d5-46ce57054ebd None admin] PATH_INFO: `/v3/users/ed513a0aabf84899951cac87303e0a8f` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:32.249195 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c885c40-209b-44bc-95d5-46ce57054ebd None admin] RBAC: Authorizing `identity:get_user(user_id=ed513a0aabf84899951cac87303e0a8f)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:32.251094 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c885c40-209b-44bc-95d5-46ce57054ebd None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:32.254816 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 57/108] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:53:32 2022] GET /identity/v3/users/ed513a0aabf84899951cac87303e0a8f => generated 289 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:32.261403 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c885c40-209b-44bc-95d5-46ce57054ebd None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:32.279610 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:32.468943 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebb0aa-dbdd-43d6-b992-3b257fff207a None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:32.497906 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:32.518636 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c885c40-209b-44bc-95d5-46ce57054ebd None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:32.551175 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f388cb-29e6-4cb5-9b31-7f59a208f76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:32.552176 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f388cb-29e6-4cb5-9b31-7f59a208f76c None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:32.555561 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f388cb-29e6-4cb5-9b31-7f59a208f76c None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:32.556378 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f388cb-29e6-4cb5-9b31-7f59a208f76c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:32.577125 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f388cb-29e6-4cb5-9b31-7f59a208f76c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:53:32.583355 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-07f388cb-29e6-4cb5-9b31-7f59a208f76c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:53:32.583355 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:53:32.583355 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:53:32.583355 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:53:32.583355 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:53:32.583355 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:53:32.583355 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:53:32.583355 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:53:32.583355 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:53:32.583355 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:53:32.583355 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:53:32.583355 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:53:32.583355 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:53:32.583355 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:53:32.583355 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:53:32.583355 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:53:32.584220 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:53:32.584220 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:53:32.584220 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:53:32.584220 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:53:32.584220 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:53:32.584220 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:53:32.584220 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:53:32.584220 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:53:32.584220 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:53:32.584220 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:53:32.584220 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:53:32.584220 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:53:32.584220 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:53:32.584220 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:53:32.584220 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:53:32.584220 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:53:32.584987 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:53:32.584987 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:53:32.584987 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:53:32.584987 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:53:32.584987 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:53:32.584987 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:53:32.584987 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:53:32.584987 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:53:32.584987 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:53:32.584987 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:53:32.584987 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:53:32.584987 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:53:32.595170 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 52/109] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:53:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:53:32.596451 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f388cb-29e6-4cb5-9b31-7f59a208f76c None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:32.600123 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f527b5-ea8e-43f0-b69c-2ccad5f24ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:32.601442 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f527b5-ea8e-43f0-b69c-2ccad5f24ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:32.601442 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f527b5-ea8e-43f0-b69c-2ccad5f24ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:32.601442 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f527b5-ea8e-43f0-b69c-2ccad5f24ee0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:32.753276 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:32.759932 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f527b5-ea8e-43f0-b69c-2ccad5f24ee0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:32.762176 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f527b5-ea8e-43f0-b69c-2ccad5f24ee0 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:32.777475 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f388cb-29e6-4cb5-9b31-7f59a208f76c None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:32.834330 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e48389d-b5a7-4900-8cff-937afec86a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:32.835009 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e48389d-b5a7-4900-8cff-937afec86a50 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:32.835295 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e48389d-b5a7-4900-8cff-937afec86a50 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:32.835575 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e48389d-b5a7-4900-8cff-937afec86a50 None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:32.836393 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e48389d-b5a7-4900-8cff-937afec86a50 None admin] RBAC: Adding query filter params (name=member) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:32.836707 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e48389d-b5a7-4900-8cff-937afec86a50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:32.842874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e48389d-b5a7-4900-8cff-937afec86a50 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:32.849608 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 53/110] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 14:53:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:32.851210 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cce0cc-43e8-4466-bc54-647f0e0c9ef0 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:32.851501 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cce0cc-43e8-4466-bc54-647f0e0c9ef0 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:32.851781 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cce0cc-43e8-4466-bc54-647f0e0c9ef0 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:32.853425 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 54/111] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:32.860994 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cce0cc-43e8-4466-bc54-647f0e0c9ef0 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:32.955884 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:32.971758 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 58/112] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:53:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 711 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:53:32.975980 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e7b33-e03e-4b4a-aa3e-5e81fb4e0acd None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:32.977317 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e7b33-e03e-4b4a-aa3e-5e81fb4e0acd None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:32.977963 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e7b33-e03e-4b4a-aa3e-5e81fb4e0acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:32.999818 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-e83e7b33-e03e-4b4a-aa3e-5e81fb4e0acd None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:33.001319 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-e83e7b33-e03e-4b4a-aa3e-5e81fb4e0acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:33.044952 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cce0cc-43e8-4466-bc54-647f0e0c9ef0 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:33.073124 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 59/113] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:33.081094 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe47ad4-23ed-4178-93d2-77f94788d898 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:33.081914 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe47ad4-23ed-4178-93d2-77f94788d898 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:33.082523 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe47ad4-23ed-4178-93d2-77f94788d898 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:33.082998 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bff9b9-07ba-4e61-a710-8ae7d22af3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:33.083655 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bff9b9-07ba-4e61-a710-8ae7d22af3d9 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:33.083936 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bff9b9-07ba-4e61-a710-8ae7d22af3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:33.084211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bff9b9-07ba-4e61-a710-8ae7d22af3d9 None admin] PATH_INFO: `/v3/users/4585768734fc4332a61c845383c1482c` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:33.085362 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 60/114] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:33.085641 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bff9b9-07ba-4e61-a710-8ae7d22af3d9 None admin] RBAC: Authorizing `identity:get_user(user_id=4585768734fc4332a61c845383c1482c)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:33.086750 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe47ad4-23ed-4178-93d2-77f94788d898 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:33.087024 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bff9b9-07ba-4e61-a710-8ae7d22af3d9 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:33.089689 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 55/115] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:53:32 2022] GET /identity/v3/users/4585768734fc4332a61c845383c1482c => generated 275 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:33.092232 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7aab5c-0cf6-4d2e-9b80-32c38cef2539 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:33.092232 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7aab5c-0cf6-4d2e-9b80-32c38cef2539 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:33.092232 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7aab5c-0cf6-4d2e-9b80-32c38cef2539 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:33.092232 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 56/116] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:33.093381 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd34c6-43f0-49d7-a90d-d9fc25aa45a7 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:33.093673 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd34c6-43f0-49d7-a90d-d9fc25aa45a7 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:33.093939 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd34c6-43f0-49d7-a90d-d9fc25aa45a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:33.119980 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-42fd34c6-43f0-49d7-a90d-d9fc25aa45a7 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:33.121188 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-42fd34c6-43f0-49d7-a90d-d9fc25aa45a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:33.135663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:33.195775 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 57/117] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:33.196783 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd34c6-43f0-49d7-a90d-d9fc25aa45a7 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:33.252317 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe47ad4-23ed-4178-93d2-77f94788d898 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:33.293220 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:33.301842 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd34c6-43f0-49d7-a90d-d9fc25aa45a7 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:33.318701 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3c2356-fbf9-4aa2-bd7a-a23422e4226a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:33.319377 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c2356-fbf9-4aa2-bd7a-a23422e4226a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:33.319664 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c2356-fbf9-4aa2-bd7a-a23422e4226a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:33.319942 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c2356-fbf9-4aa2-bd7a-a23422e4226a None admin] PATH_INFO: `/v3/projects/3b402af2708b41df849963f81e65797e` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:33.321101 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3c2356-fbf9-4aa2-bd7a-a23422e4226a None admin] RBAC: Authorizing `identity:get_project(project_id=3b402af2708b41df849963f81e65797e)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:33.322572 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3c2356-fbf9-4aa2-bd7a-a23422e4226a None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:33.324246 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 61/118] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:53:33 2022] GET /identity/v3/projects/3b402af2708b41df849963f81e65797e => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:33.325365 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f90645-2d53-4152-a26b-31c926134033 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:33.325651 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f90645-2d53-4152-a26b-31c926134033 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:33.325929 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f90645-2d53-4152-a26b-31c926134033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:33.345693 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-e6f90645-2d53-4152-a26b-31c926134033 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:33.347060 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-e6f90645-2d53-4152-a26b-31c926134033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:33.362106 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97383d8c-06d8-4ea1-894d-a11116b6d228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:33.362829 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97383d8c-06d8-4ea1-894d-a11116b6d228 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:33.363115 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97383d8c-06d8-4ea1-894d-a11116b6d228 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:33.363489 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97383d8c-06d8-4ea1-894d-a11116b6d228 None admin] PATH_INFO: `/v3/projects/3b402af2708b41df849963f81e65797e` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:33.367941 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97383d8c-06d8-4ea1-894d-a11116b6d228 None admin] RBAC: Authorizing `identity:get_project(project_id=3b402af2708b41df849963f81e65797e)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:33.370783 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97383d8c-06d8-4ea1-894d-a11116b6d228 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:33.377114 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 58/119] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:53:33 2022] GET /identity/v3/projects/3b402af2708b41df849963f81e65797e => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:33.378099 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97383d8c-06d8-4ea1-894d-a11116b6d228 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:33.402076 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:33.451100 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 62/120] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:33.452202 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f90645-2d53-4152-a26b-31c926134033 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:33.495395 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97383d8c-06d8-4ea1-894d-a11116b6d228 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:33.534154 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41445358-d033-4da2-b76d-813655636ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:33.534775 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41445358-d033-4da2-b76d-813655636ded None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:33.535088 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41445358-d033-4da2-b76d-813655636ded None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:33.535369 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41445358-d033-4da2-b76d-813655636ded None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:33.536111 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41445358-d033-4da2-b76d-813655636ded None admin] RBAC: Adding query filter params (role.id=06e98804662441c88b8871c32b9468aa, scope.project.id=3b402af2708b41df849963f81e65797e, user.id=ed513a0aabf84899951cac87303e0a8f) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:33.536426 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41445358-d033-4da2-b76d-813655636ded None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:33.537897 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41445358-d033-4da2-b76d-813655636ded None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:33.545803 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 59/121] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:53:33 2022] GET /identity/v3/role_assignments?user.id=ed513a0aabf84899951cac87303e0a8f&scope.project.id=3b402af2708b41df849963f81e65797e&role.id=06e98804662441c88b8871c32b9468aa => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:33.548365 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41445358-d033-4da2-b76d-813655636ded None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:33.587707 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f90645-2d53-4152-a26b-31c926134033 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:33.631253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:33.656008 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc2ff3d-6d03-4fe7-8e57-25bfa9f09a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:33.659051 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2ff3d-6d03-4fe7-8e57-25bfa9f09a0c None admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:33.659339 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2ff3d-6d03-4fe7-8e57-25bfa9f09a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:33.659620 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2ff3d-6d03-4fe7-8e57-25bfa9f09a0c None admin] PATH_INFO: `/v3/projects/3b402af2708b41df849963f81e65797e/users/4585768734fc4332a61c845383c1482c/roles/c9da950c75da4939b8bd649c11015650` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:33.665721 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc2ff3d-6d03-4fe7-8e57-25bfa9f09a0c None admin] RBAC: Authorizing `identity:create_grant(project_id=3b402af2708b41df849963f81e65797e, user_id=4585768734fc4332a61c845383c1482c, role_id=c9da950c75da4939b8bd649c11015650)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:33.668510 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc2ff3d-6d03-4fe7-8e57-25bfa9f09a0c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:33.694962 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 63/122] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 14:53:33 2022] PUT /identity/v3/projects/3b402af2708b41df849963f81e65797e/users/4585768734fc4332a61c845383c1482c/roles/c9da950c75da4939b8bd649c11015650 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:53:33.696008 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc2ff3d-6d03-4fe7-8e57-25bfa9f09a0c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:33.733099 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:33.764058 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41445358-d033-4da2-b76d-813655636ded None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:33.814595 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df46c7d-b2d0-4abc-ac5e-eba0c1e8c25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:33.815287 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df46c7d-b2d0-4abc-ac5e-eba0c1e8c25f None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:33.815579 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df46c7d-b2d0-4abc-ac5e-eba0c1e8c25f None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:33.815916 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df46c7d-b2d0-4abc-ac5e-eba0c1e8c25f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:33.832747 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df46c7d-b2d0-4abc-ac5e-eba0c1e8c25f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:53:33.839734 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-9df46c7d-b2d0-4abc-ac5e-eba0c1e8c25f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:53:33.839734 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:53:33.839734 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:53:33.839734 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:53:33.839734 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:53:33.839734 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:53:33.839734 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:53:33.839734 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:53:33.839734 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:53:33.839734 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:53:33.839734 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:53:33.839734 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:53:33.839734 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:53:33.839734 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:53:33.839734 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:53:33.839734 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:53:33.840645 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:53:33.840645 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:53:33.840645 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:53:33.840645 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:53:33.840645 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:53:33.840645 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:53:33.840645 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:53:33.840645 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:53:33.840645 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:53:33.840645 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:53:33.840645 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:53:33.840645 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:53:33.840645 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:53:33.840645 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:53:33.840645 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:53:33.840645 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:53:33.841419 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:53:33.841419 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:53:33.841419 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:53:33.841419 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:53:33.841419 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:53:33.841419 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:53:33.841419 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:53:33.841419 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:53:33.841419 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:53:33.841419 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:53:33.841419 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:53:33.841419 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:53:33.841419 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 60/123] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:53:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:53:33.842085 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df46c7d-b2d0-4abc-ac5e-eba0c1e8c25f None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:33.884843 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc2ff3d-6d03-4fe7-8e57-25bfa9f09a0c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:33.936325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c347c9-b6dc-479b-a50e-82095f0cc4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:33.937482 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c347c9-b6dc-479b-a50e-82095f0cc4dd None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:33.937972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c347c9-b6dc-479b-a50e-82095f0cc4dd None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:33.938286 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c347c9-b6dc-479b-a50e-82095f0cc4dd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:33.953078 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c347c9-b6dc-479b-a50e-82095f0cc4dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:53:33.961708 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-a8c347c9-b6dc-479b-a50e-82095f0cc4dd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:53:33.961708 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:53:33.961708 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:53:33.961708 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:53:33.961708 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:53:33.961708 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:53:33.961708 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:53:33.961708 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:53:33.961708 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:53:33.961708 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:53:33.961708 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:53:33.961708 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:53:33.961708 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:53:33.961708 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:53:33.961708 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:53:33.961708 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:53:33.962574 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:53:33.962574 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:53:33.962574 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:53:33.962574 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:53:33.962574 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:53:33.962574 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:53:33.962574 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:53:33.962574 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:53:33.962574 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:53:33.962574 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:53:33.962574 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:53:33.962574 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:53:33.962574 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:53:33.962574 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:53:33.962574 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:53:33.962574 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:53:33.963396 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:53:33.963396 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:53:33.963396 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:53:33.963396 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:53:33.963396 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:53:33.963396 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:53:33.963396 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:53:33.963396 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:53:33.963396 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:53:33.963396 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:53:33.963396 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:53:33.963396 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:53:33.963396 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 64/124] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 14:53:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:53:33.964042 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c347c9-b6dc-479b-a50e-82095f0cc4dd None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:34.006459 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df46c7d-b2d0-4abc-ac5e-eba0c1e8c25f None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:34.030952 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:34.051351 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa280e-89bd-477a-bdb4-9eb9044dc745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:34.051995 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa280e-89bd-477a-bdb4-9eb9044dc745 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:34.052283 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa280e-89bd-477a-bdb4-9eb9044dc745 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:34.052684 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa280e-89bd-477a-bdb4-9eb9044dc745 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:34.080237 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:34.086226 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aa280e-89bd-477a-bdb4-9eb9044dc745 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:53:34.101969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-67aa280e-89bd-477a-bdb4-9eb9044dc745 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:53:34.101969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:53:34.101969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:53:34.101969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:53:34.101969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:53:34.101969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:53:34.101969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:53:34.101969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:53:34.101969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:53:34.101969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:53:34.101969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:53:34.101969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:53:34.101969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:53:34.101969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:53:34.101969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:53:34.101969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:53:34.103105 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:53:34.103105 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:53:34.103105 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:53:34.103105 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:53:34.103105 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:53:34.103105 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:53:34.103105 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:53:34.103105 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:53:34.103105 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:53:34.103105 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:53:34.103105 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:53:34.103105 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:53:34.103105 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:53:34.103105 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:53:34.103105 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:53:34.103105 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:53:34.104069 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:53:34.104069 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:53:34.104069 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:53:34.104069 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:53:34.104069 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:53:34.104069 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:53:34.104069 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:53:34.104069 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:53:34.104069 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:53:34.104069 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:53:34.104069 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:53:34.104069 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:53:34.104069 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 61/125] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:53:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:53:34.104732 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa280e-89bd-477a-bdb4-9eb9044dc745 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:34.114639 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c347c9-b6dc-479b-a50e-82095f0cc4dd None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:34.154839 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c70d8-b7e1-46a0-89e1-a6c057ed9c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:34.155732 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c70d8-b7e1-46a0-89e1-a6c057ed9c9d None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:34.156162 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c70d8-b7e1-46a0-89e1-a6c057ed9c9d None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:34.156582 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c70d8-b7e1-46a0-89e1-a6c057ed9c9d None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:34.158175 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c70d8-b7e1-46a0-89e1-a6c057ed9c9d None admin] RBAC: Adding query filter params (name=member) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:34.158626 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c70d8-b7e1-46a0-89e1-a6c057ed9c9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:34.160412 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c70d8-b7e1-46a0-89e1-a6c057ed9c9d None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:34.171562 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 65/126] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 14:53:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:34.173149 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c70d8-b7e1-46a0-89e1-a6c057ed9c9d None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:34.277558 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa280e-89bd-477a-bdb4-9eb9044dc745 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:34.316507 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369b34d2-f2a1-4943-9291-c5cd4c25dfae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:34.317392 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b34d2-f2a1-4943-9291-c5cd4c25dfae None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:34.317822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b34d2-f2a1-4943-9291-c5cd4c25dfae None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:34.318244 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b34d2-f2a1-4943-9291-c5cd4c25dfae None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:34.319146 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369b34d2-f2a1-4943-9291-c5cd4c25dfae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:34.319600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369b34d2-f2a1-4943-9291-c5cd4c25dfae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:34.321307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369b34d2-f2a1-4943-9291-c5cd4c25dfae None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:34.330615 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 62/127] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:53:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:34.331671 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369b34d2-f2a1-4943-9291-c5cd4c25dfae None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:34.401377 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:34.404732 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c70d8-b7e1-46a0-89e1-a6c057ed9c9d None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:34.407658 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:34.445549 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369b34d2-f2a1-4943-9291-c5cd4c25dfae None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:34.482260 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66403d9-9ff3-48b9-a839-85eda7737fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:34.483170 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66403d9-9ff3-48b9-a839-85eda7737fde None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:34.483599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66403d9-9ff3-48b9-a839-85eda7737fde None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:34.484021 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66403d9-9ff3-48b9-a839-85eda7737fde None admin] PATH_INFO: `/v3/users/4585768734fc4332a61c845383c1482c` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:34.486177 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66403d9-9ff3-48b9-a839-85eda7737fde None admin] RBAC: Authorizing `identity:get_user(user_id=4585768734fc4332a61c845383c1482c)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:34.488387 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66403d9-9ff3-48b9-a839-85eda7737fde None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:34.490562 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 63/128] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:53:34 2022] GET /identity/v3/users/4585768734fc4332a61c845383c1482c => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:34.491698 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66403d9-9ff3-48b9-a839-85eda7737fde None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:34.505310 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98735f92-d2ef-4e7b-a8e5-3ee265ff8055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:34.505736 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98735f92-d2ef-4e7b-a8e5-3ee265ff8055 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:34.506021 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98735f92-d2ef-4e7b-a8e5-3ee265ff8055 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:34.506301 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98735f92-d2ef-4e7b-a8e5-3ee265ff8055 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:34.507028 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98735f92-d2ef-4e7b-a8e5-3ee265ff8055 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:34.507339 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98735f92-d2ef-4e7b-a8e5-3ee265ff8055 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:34.508822 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98735f92-d2ef-4e7b-a8e5-3ee265ff8055 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:34.543700 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 66/129] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:53:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:34.544643 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98735f92-d2ef-4e7b-a8e5-3ee265ff8055 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:34.609291 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66403d9-9ff3-48b9-a839-85eda7737fde None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:34.642033 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:34.649739 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7cd079-50d1-42e2-9fc3-88f12f4bca40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:34.650603 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7cd079-50d1-42e2-9fc3-88f12f4bca40 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:34.651082 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7cd079-50d1-42e2-9fc3-88f12f4bca40 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:34.651558 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7cd079-50d1-42e2-9fc3-88f12f4bca40 None admin] PATH_INFO: `/v3/users/4585768734fc4332a61c845383c1482c` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:34.652933 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7cd079-50d1-42e2-9fc3-88f12f4bca40 None admin] RBAC: Authorizing `identity:get_user(user_id=4585768734fc4332a61c845383c1482c)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:34.655187 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7cd079-50d1-42e2-9fc3-88f12f4bca40 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:34.658231 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 64/130] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:53:34 2022] GET /identity/v3/users/4585768734fc4332a61c845383c1482c => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:34.659354 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7cd079-50d1-42e2-9fc3-88f12f4bca40 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:34.720693 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98735f92-d2ef-4e7b-a8e5-3ee265ff8055 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:34.759534 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5afcba7-e8ef-496b-b8bb-8228a4bd51c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:34.760203 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afcba7-e8ef-496b-b8bb-8228a4bd51c9 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:34.760489 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afcba7-e8ef-496b-b8bb-8228a4bd51c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:34.760772 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afcba7-e8ef-496b-b8bb-8228a4bd51c9 None admin] PATH_INFO: `/v3/projects/3dc3bfe113c841f098a02b2b1ec28edc` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:34.762016 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5afcba7-e8ef-496b-b8bb-8228a4bd51c9 None admin] RBAC: Authorizing `identity:get_project(project_id=3dc3bfe113c841f098a02b2b1ec28edc)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:34.763857 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5afcba7-e8ef-496b-b8bb-8228a4bd51c9 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:34.765800 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 67/131] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:53:34 2022] GET /identity/v3/projects/3dc3bfe113c841f098a02b2b1ec28edc => generated 312 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:34.768247 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c073be-a1b8-4bbd-96e8-43d2e9deb31f None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:34.768539 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c073be-a1b8-4bbd-96e8-43d2e9deb31f None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:34.768820 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c073be-a1b8-4bbd-96e8-43d2e9deb31f None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:34.770194 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 68/132] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:34.775546 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c073be-a1b8-4bbd-96e8-43d2e9deb31f None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:34.779466 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7cd079-50d1-42e2-9fc3-88f12f4bca40 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:34.809808 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:34.825734 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a22467-a41f-4d62-893c-0beed57dd3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:34.825734 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a22467-a41f-4d62-893c-0beed57dd3ff None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:34.825734 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a22467-a41f-4d62-893c-0beed57dd3ff None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:34.825734 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a22467-a41f-4d62-893c-0beed57dd3ff None admin] PATH_INFO: `/v3/users/53878dbb519c4b3d8858375928f1c2d4` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:34.826118 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a22467-a41f-4d62-893c-0beed57dd3ff None admin] RBAC: Authorizing `identity:get_user(user_id=53878dbb519c4b3d8858375928f1c2d4)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:34.826118 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a22467-a41f-4d62-893c-0beed57dd3ff None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:34.826118 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 65/133] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:53:34 2022] GET /identity/v3/users/53878dbb519c4b3d8858375928f1c2d4 => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:34.826510 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a22467-a41f-4d62-893c-0beed57dd3ff None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:34.904064 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c073be-a1b8-4bbd-96e8-43d2e9deb31f None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:34.908132 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:34.950250 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a22467-a41f-4d62-893c-0beed57dd3ff None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:34.954170 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9ba05-0798-4201-9960-416b1c1caa74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:34.954943 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9ba05-0798-4201-9960-416b1c1caa74 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:34.955303 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9ba05-0798-4201-9960-416b1c1caa74 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:34.955707 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9ba05-0798-4201-9960-416b1c1caa74 None admin] PATH_INFO: `/v3/projects/3b402af2708b41df849963f81e65797e` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:34.956969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9ba05-0798-4201-9960-416b1c1caa74 None admin] RBAC: Authorizing `identity:get_project(project_id=3b402af2708b41df849963f81e65797e)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:34.959095 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9ba05-0798-4201-9960-416b1c1caa74 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:34.961706 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 69/134] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:53:34 2022] GET /identity/v3/projects/3b402af2708b41df849963f81e65797e => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:34.963276 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27602f2b-c1c9-4310-bc76-e8fe6acba1e0 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:34.963636 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27602f2b-c1c9-4310-bc76-e8fe6acba1e0 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:34.963988 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27602f2b-c1c9-4310-bc76-e8fe6acba1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:34.989780 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-27602f2b-c1c9-4310-bc76-e8fe6acba1e0 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:34.991059 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-27602f2b-c1c9-4310-bc76-e8fe6acba1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:35.001333 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297cac1d-51d3-48d5-b436-fbe03269e2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:35.002016 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297cac1d-51d3-48d5-b436-fbe03269e2bf None admin] REQUEST_METHOD: `PUT` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:35.002377 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297cac1d-51d3-48d5-b436-fbe03269e2bf None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:35.002668 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297cac1d-51d3-48d5-b436-fbe03269e2bf None admin] PATH_INFO: `/v3/projects/3dc3bfe113c841f098a02b2b1ec28edc/users/4585768734fc4332a61c845383c1482c/roles/c9da950c75da4939b8bd649c11015650` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:35.006279 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297cac1d-51d3-48d5-b436-fbe03269e2bf None admin] RBAC: Authorizing `identity:create_grant(project_id=3dc3bfe113c841f098a02b2b1ec28edc, user_id=4585768734fc4332a61c845383c1482c, role_id=c9da950c75da4939b8bd649c11015650)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:35.007783 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297cac1d-51d3-48d5-b436-fbe03269e2bf None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:35.028343 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 66/135] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 14:53:34 2022] PUT /identity/v3/projects/3dc3bfe113c841f098a02b2b1ec28edc/users/4585768734fc4332a61c845383c1482c/roles/c9da950c75da4939b8bd649c11015650 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:53:35.035845 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297cac1d-51d3-48d5-b436-fbe03269e2bf None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:35.084414 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 70/136] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:35.088856 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27602f2b-c1c9-4310-bc76-e8fe6acba1e0 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:35.145737 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:35.233863 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27602f2b-c1c9-4310-bc76-e8fe6acba1e0 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:35.272640 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e3886-ad49-4d8e-ab6b-cdc8d7e2fa86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:35.273674 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e3886-ad49-4d8e-ab6b-cdc8d7e2fa86 None admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:35.274184 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e3886-ad49-4d8e-ab6b-cdc8d7e2fa86 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:35.274546 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e3886-ad49-4d8e-ab6b-cdc8d7e2fa86 None admin] PATH_INFO: `/v3/projects/3b402af2708b41df849963f81e65797e/users/4585768734fc4332a61c845383c1482c/roles/d1818eb7f2bd467fbf75bad5cc8e0aea` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:35.276926 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e3886-ad49-4d8e-ab6b-cdc8d7e2fa86 None admin] RBAC: Authorizing `identity:create_grant(project_id=3b402af2708b41df849963f81e65797e, user_id=4585768734fc4332a61c845383c1482c, role_id=d1818eb7f2bd467fbf75bad5cc8e0aea)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:35.278539 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e3886-ad49-4d8e-ab6b-cdc8d7e2fa86 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:35.288725 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297cac1d-51d3-48d5-b436-fbe03269e2bf None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:35.298495 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 71/137] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 14:53:35 2022] PUT /identity/v3/projects/3b402af2708b41df849963f81e65797e/users/4585768734fc4332a61c845383c1482c/roles/d1818eb7f2bd467fbf75bad5cc8e0aea => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:53:35.341990 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:35.356217 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af93029-dbfe-4378-8d96-c78dcc52b2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:35.357179 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af93029-dbfe-4378-8d96-c78dcc52b2c7 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:35.357631 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af93029-dbfe-4378-8d96-c78dcc52b2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:35.358122 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af93029-dbfe-4378-8d96-c78dcc52b2c7 None admin] PATH_INFO: `/v3/projects/3b402af2708b41df849963f81e65797e` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:35.359791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af93029-dbfe-4378-8d96-c78dcc52b2c7 None admin] RBAC: Authorizing `identity:get_project(project_id=3b402af2708b41df849963f81e65797e)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:35.362244 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af93029-dbfe-4378-8d96-c78dcc52b2c7 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:35.364210 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 67/138] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:53:35 2022] GET /identity/v3/projects/3b402af2708b41df849963f81e65797e => generated 298 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:35.370388 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e3886-ad49-4d8e-ab6b-cdc8d7e2fa86 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:35.479899 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:35.491860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e3886-ad49-4d8e-ab6b-cdc8d7e2fa86 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:35.530059 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af93029-dbfe-4378-8d96-c78dcc52b2c7 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:35.532100 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f1aa1-c64a-46c3-86f3-5c2276f0dbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:35.532800 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f1aa1-c64a-46c3-86f3-5c2276f0dbfd None admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:35.533166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f1aa1-c64a-46c3-86f3-5c2276f0dbfd None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:35.533684 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f1aa1-c64a-46c3-86f3-5c2276f0dbfd None admin] PATH_INFO: `/v3/projects/3b402af2708b41df849963f81e65797e/users/53878dbb519c4b3d8858375928f1c2d4/roles/99e8fe7df8e7446680a81d4eaaa1cb68` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:35.536766 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f1aa1-c64a-46c3-86f3-5c2276f0dbfd None admin] RBAC: Authorizing `identity:create_grant(project_id=3b402af2708b41df849963f81e65797e, user_id=53878dbb519c4b3d8858375928f1c2d4, role_id=99e8fe7df8e7446680a81d4eaaa1cb68)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:35.538403 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f1aa1-c64a-46c3-86f3-5c2276f0dbfd None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:35.556524 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 72/139] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 14:53:35 2022] PUT /identity/v3/projects/3b402af2708b41df849963f81e65797e/users/53878dbb519c4b3d8858375928f1c2d4/roles/99e8fe7df8e7446680a81d4eaaa1cb68 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:53:35.690600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af93029-dbfe-4378-8d96-c78dcc52b2c7 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:35.732426 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c51b2e-5e3d-4f52-90e2-d0063a79f434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:35.733071 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c51b2e-5e3d-4f52-90e2-d0063a79f434 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:35.733351 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c51b2e-5e3d-4f52-90e2-d0063a79f434 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:35.733625 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c51b2e-5e3d-4f52-90e2-d0063a79f434 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:35.734825 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c51b2e-5e3d-4f52-90e2-d0063a79f434 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:35.736644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c51b2e-5e3d-4f52-90e2-d0063a79f434 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:35.738635 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 68/140] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:53:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:53:35.744324 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f1aa1-c64a-46c3-86f3-5c2276f0dbfd None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:35.880455 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f1aa1-c64a-46c3-86f3-5c2276f0dbfd None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:35.918739 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc12cbd-0a98-4e82-ba0f-d634d5f0d5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:35.919379 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc12cbd-0a98-4e82-ba0f-d634d5f0d5d4 None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:35.919644 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc12cbd-0a98-4e82-ba0f-d634d5f0d5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:35.920052 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc12cbd-0a98-4e82-ba0f-d634d5f0d5d4 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:35.920734 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc12cbd-0a98-4e82-ba0f-d634d5f0d5d4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:35.922197 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc12cbd-0a98-4e82-ba0f-d634d5f0d5d4 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:35.961704 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 73/141] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 14:53:35 2022] POST /identity/v3/users => generated 283 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:53:36.446447 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b4b7a2-7b09-4210-80f8-57f84bc2f7d0 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:36.447086 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b4b7a2-7b09-4210-80f8-57f84bc2f7d0 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:36.447086 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b4b7a2-7b09-4210-80f8-57f84bc2f7d0 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:36.448389 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 69/142] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:36.459441 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613aa670-05bc-4780-96ab-933326154ec3 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:36.459778 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613aa670-05bc-4780-96ab-933326154ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:36.460014 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613aa670-05bc-4780-96ab-933326154ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:36.477527 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:36.486724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-613aa670-05bc-4780-96ab-933326154ec3 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:36.488025 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-613aa670-05bc-4780-96ab-933326154ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:36.578277 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 74/143] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:36.738144 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd78877-48d6-407a-a7df-1d8a2f0d7c09 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:36.738472 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd78877-48d6-407a-a7df-1d8a2f0d7c09 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:36.738714 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd78877-48d6-407a-a7df-1d8a2f0d7c09 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:36.740344 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 70/144] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:36.745586 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ffe136-4640-4e9e-af8a-f6d418b1fbb9 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:36.745875 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ffe136-4640-4e9e-af8a-f6d418b1fbb9 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:36.746155 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ffe136-4640-4e9e-af8a-f6d418b1fbb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:36.767910 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-34ffe136-4640-4e9e-af8a-f6d418b1fbb9 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:36.769125 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-34ffe136-4640-4e9e-af8a-f6d418b1fbb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:36.868442 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 75/145] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:37.004898 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd78877-48d6-407a-a7df-1d8a2f0d7c09 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:37.078400 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:37.123722 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd78877-48d6-407a-a7df-1d8a2f0d7c09 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:37.162160 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf2b707-c710-493e-a568-8656e21c654a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:37.162823 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2b707-c710-493e-a568-8656e21c654a None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:37.163106 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2b707-c710-493e-a568-8656e21c654a None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:37.163384 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2b707-c710-493e-a568-8656e21c654a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:37.173866 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf2b707-c710-493e-a568-8656e21c654a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:53:37.179285 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-2bf2b707-c710-493e-a568-8656e21c654a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:53:37.179285 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:53:37.179285 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:53:37.179285 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:53:37.179285 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:53:37.179285 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:53:37.179285 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:53:37.179285 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:53:37.179285 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:53:37.179285 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:53:37.179285 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:53:37.179285 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:53:37.179285 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:53:37.179285 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:53:37.179285 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:53:37.179285 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:53:37.180264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:53:37.180264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:53:37.180264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:53:37.180264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:53:37.180264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:53:37.180264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:53:37.180264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:53:37.180264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:53:37.180264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:53:37.180264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:53:37.180264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:53:37.180264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:53:37.180264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:53:37.180264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:53:37.180264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:53:37.180264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:53:37.181065 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:53:37.181065 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:53:37.181065 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:53:37.181065 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:53:37.181065 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:53:37.181065 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:53:37.181065 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:53:37.181065 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:53:37.181065 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:53:37.181065 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:53:37.181065 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:53:37.181065 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:53:37.181065 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 71/146] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:53:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:53:37.188794 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ffe136-4640-4e9e-af8a-f6d418b1fbb9 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:37.266080 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf2b707-c710-493e-a568-8656e21c654a None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:37.284074 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:37.335507 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ffe136-4640-4e9e-af8a-f6d418b1fbb9 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:37.380786 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d76da-3700-4b00-bd9f-c8d840e19960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:37.381415 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d76da-3700-4b00-bd9f-c8d840e19960 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:37.381697 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d76da-3700-4b00-bd9f-c8d840e19960 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:37.381972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d76da-3700-4b00-bd9f-c8d840e19960 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:37.382717 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d76da-3700-4b00-bd9f-c8d840e19960 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:37.383057 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d76da-3700-4b00-bd9f-c8d840e19960 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:37.384831 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d76da-3700-4b00-bd9f-c8d840e19960 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:37.395688 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 76/147] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 14:53:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:37.402843 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d76da-3700-4b00-bd9f-c8d840e19960 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:37.411686 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf2b707-c710-493e-a568-8656e21c654a None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:37.440200 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:37.473118 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac097a-20ff-4236-b058-ce49271287e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:37.473871 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac097a-20ff-4236-b058-ce49271287e2 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:37.474121 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac097a-20ff-4236-b058-ce49271287e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:37.474407 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac097a-20ff-4236-b058-ce49271287e2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:37.495232 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac097a-20ff-4236-b058-ce49271287e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:53:37.507237 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-96ac097a-20ff-4236-b058-ce49271287e2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:53:37.507237 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:53:37.507237 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:53:37.507237 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:53:37.507237 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:53:37.507237 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:53:37.507237 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:53:37.507237 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:53:37.507237 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:53:37.507237 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:53:37.507237 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:53:37.507237 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:53:37.507237 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:53:37.507237 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:53:37.507237 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:53:37.507237 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:53:37.508151 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:53:37.508151 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:53:37.508151 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:53:37.508151 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:53:37.508151 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:53:37.508151 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:53:37.508151 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:53:37.508151 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:53:37.508151 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:53:37.508151 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:53:37.508151 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:53:37.508151 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:53:37.508151 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:53:37.508151 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:53:37.508151 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:53:37.508151 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:53:37.508925 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:53:37.508925 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:53:37.508925 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:53:37.508925 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:53:37.508925 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:53:37.508925 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:53:37.508925 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:53:37.508925 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:53:37.508925 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:53:37.508925 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:53:37.508925 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:53:37.508925 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:53:37.508925 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 72/148] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:53:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:53:37.517709 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac097a-20ff-4236-b058-ce49271287e2 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:37.620083 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d76da-3700-4b00-bd9f-c8d840e19960 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:37.692095 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:37.706689 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b14b0-847d-457d-8bca-4ab91e89d588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:37.707376 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b14b0-847d-457d-8bca-4ab91e89d588 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:37.707658 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b14b0-847d-457d-8bca-4ab91e89d588 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:37.708651 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b14b0-847d-457d-8bca-4ab91e89d588 None admin] PATH_INFO: `/v3/users/ed513a0aabf84899951cac87303e0a8f` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:37.711733 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7b14b0-847d-457d-8bca-4ab91e89d588 None admin] RBAC: Authorizing `identity:get_user(user_id=ed513a0aabf84899951cac87303e0a8f)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:37.713233 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7b14b0-847d-457d-8bca-4ab91e89d588 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:37.723587 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 77/149] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:53:37 2022] GET /identity/v3/users/ed513a0aabf84899951cac87303e0a8f => generated 289 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:37.741606 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac097a-20ff-4236-b058-ce49271287e2 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:37.760226 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b14b0-847d-457d-8bca-4ab91e89d588 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:37.797133 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d64ea32-830d-468e-bbb5-894cc2ceb5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:37.798231 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d64ea32-830d-468e-bbb5-894cc2ceb5dc None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:37.798515 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d64ea32-830d-468e-bbb5-894cc2ceb5dc None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:37.798786 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d64ea32-830d-468e-bbb5-894cc2ceb5dc None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:37.799491 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d64ea32-830d-468e-bbb5-894cc2ceb5dc None admin] RBAC: Adding query filter params (name=member) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:37.799792 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d64ea32-830d-468e-bbb5-894cc2ceb5dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:37.801194 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d64ea32-830d-468e-bbb5-894cc2ceb5dc None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:37.818377 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 73/150] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 14:53:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:37.824513 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d64ea32-830d-468e-bbb5-894cc2ceb5dc None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:37.900252 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:37.961653 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b14b0-847d-457d-8bca-4ab91e89d588 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:37.978335 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d64ea32-830d-468e-bbb5-894cc2ceb5dc None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:38.028147 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306e0ba-f311-4fe0-9979-3d45439ba0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:38.028803 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306e0ba-f311-4fe0-9979-3d45439ba0ff None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:38.029137 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306e0ba-f311-4fe0-9979-3d45439ba0ff None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:38.029355 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306e0ba-f311-4fe0-9979-3d45439ba0ff None admin] PATH_INFO: `/v3/users/4585768734fc4332a61c845383c1482c` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:38.030299 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993e4afd-25b3-4f16-b245-c2cf3a18fb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:38.030971 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e4afd-25b3-4f16-b245-c2cf3a18fb93 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:38.031253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e4afd-25b3-4f16-b245-c2cf3a18fb93 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:38.031526 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e4afd-25b3-4f16-b245-c2cf3a18fb93 None admin] PATH_INFO: `/v3/projects/3b402af2708b41df849963f81e65797e` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:38.033234 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993e4afd-25b3-4f16-b245-c2cf3a18fb93 None admin] RBAC: Authorizing `identity:get_project(project_id=3b402af2708b41df849963f81e65797e)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:38.033447 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5306e0ba-f311-4fe0-9979-3d45439ba0ff None admin] RBAC: Authorizing `identity:get_user(user_id=4585768734fc4332a61c845383c1482c)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:38.034710 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993e4afd-25b3-4f16-b245-c2cf3a18fb93 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:38.034959 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5306e0ba-f311-4fe0-9979-3d45439ba0ff None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:38.036738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 78/151] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:53:37 2022] GET /identity/v3/projects/3b402af2708b41df849963f81e65797e => generated 298 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:53:38.036945 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 74/152] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:53:37 2022] GET /identity/v3/users/4585768734fc4332a61c845383c1482c => generated 275 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:53:38.042285 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993e4afd-25b3-4f16-b245-c2cf3a18fb93 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:38.042774 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306e0ba-f311-4fe0-9979-3d45439ba0ff None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:38.143983 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:38.182880 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306e0ba-f311-4fe0-9979-3d45439ba0ff None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:38.199197 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:38.220230 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993e4afd-25b3-4f16-b245-c2cf3a18fb93 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:38.233515 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343365f0-1828-4da6-a095-6c1540c748b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:38.234200 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343365f0-1828-4da6-a095-6c1540c748b8 None admin] REQUEST_METHOD: `PUT` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:38.234531 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343365f0-1828-4da6-a095-6c1540c748b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:38.234834 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343365f0-1828-4da6-a095-6c1540c748b8 None admin] PATH_INFO: `/v3/projects/3b402af2708b41df849963f81e65797e/users/ed513a0aabf84899951cac87303e0a8f/roles/06e98804662441c88b8871c32b9468aa` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:38.240186 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343365f0-1828-4da6-a095-6c1540c748b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=3b402af2708b41df849963f81e65797e, user_id=ed513a0aabf84899951cac87303e0a8f, role_id=06e98804662441c88b8871c32b9468aa)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:38.241604 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343365f0-1828-4da6-a095-6c1540c748b8 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:38.265120 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 75/153] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 14:53:38 2022] PUT /identity/v3/projects/3b402af2708b41df849963f81e65797e/users/ed513a0aabf84899951cac87303e0a8f/roles/06e98804662441c88b8871c32b9468aa => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 14:53:38.283575 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a438ffd8-36f3-41cd-8b98-04716436f9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:38.284217 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438ffd8-36f3-41cd-8b98-04716436f9ab None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:38.284500 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438ffd8-36f3-41cd-8b98-04716436f9ab None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:38.284778 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438ffd8-36f3-41cd-8b98-04716436f9ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:38.386129 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a438ffd8-36f3-41cd-8b98-04716436f9ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:38.387677 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a438ffd8-36f3-41cd-8b98-04716436f9ab None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:38.478692 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:38.539224 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 79/154] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:53:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:53:38.545324 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343365f0-1828-4da6-a095-6c1540c748b8 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:38.673909 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57f7e8-b2b9-4ca6-bbb4-b7e62d4dc75d None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:38.674298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57f7e8-b2b9-4ca6-bbb4-b7e62d4dc75d None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:38.674478 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57f7e8-b2b9-4ca6-bbb4-b7e62d4dc75d None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:38.676055 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 80/155] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:53:38.684205 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a9a0b-a935-440e-a5af-ec74057ac105 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:38.684491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a9a0b-a935-440e-a5af-ec74057ac105 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:38.684767 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a9a0b-a935-440e-a5af-ec74057ac105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:38.689574 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343365f0-1828-4da6-a095-6c1540c748b8 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:38.717158 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-3f7a9a0b-a935-440e-a5af-ec74057ac105 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:38.718372 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-3f7a9a0b-a935-440e-a5af-ec74057ac105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:38.738066 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e2705f-c2bd-4525-86f0-7b42761bac84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:38.738655 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2705f-c2bd-4525-86f0-7b42761bac84 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:38.738970 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2705f-c2bd-4525-86f0-7b42761bac84 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:38.739245 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2705f-c2bd-4525-86f0-7b42761bac84 None admin] PATH_INFO: `/v3/projects/3b402af2708b41df849963f81e65797e` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:38.742195 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e2705f-c2bd-4525-86f0-7b42761bac84 None admin] RBAC: Authorizing `identity:get_project(project_id=3b402af2708b41df849963f81e65797e)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:38.746549 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e2705f-c2bd-4525-86f0-7b42761bac84 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:38.755681 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 76/156] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:53:38 2022] GET /identity/v3/projects/3b402af2708b41df849963f81e65797e => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:38.762048 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e2705f-c2bd-4525-86f0-7b42761bac84 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:38.832529 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:38.872564 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 81/157] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:38.875057 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f400f57-a97f-42f7-8a24-687fba52afdb None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:38.877068 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f400f57-a97f-42f7-8a24-687fba52afdb None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:38.877571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f400f57-a97f-42f7-8a24-687fba52afdb None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:38.880319 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 82/158] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:38.890508 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437ac6f-3db8-410c-acbf-60e7334f5a4d None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:38.891135 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437ac6f-3db8-410c-acbf-60e7334f5a4d None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:38.892185 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437ac6f-3db8-410c-acbf-60e7334f5a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:38.922518 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-0437ac6f-3db8-410c-acbf-60e7334f5a4d None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:38.925774 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-0437ac6f-3db8-410c-acbf-60e7334f5a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:38.972046 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e2705f-c2bd-4525-86f0-7b42761bac84 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:39.050091 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a0335-022e-4aa7-96ea-534c9d5903e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:39.050387 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a0335-022e-4aa7-96ea-534c9d5903e9 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:39.050467 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a0335-022e-4aa7-96ea-534c9d5903e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:39.050734 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a0335-022e-4aa7-96ea-534c9d5903e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:39.051497 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a0335-022e-4aa7-96ea-534c9d5903e9 None admin] RBAC: Adding query filter params (role.id=c9da950c75da4939b8bd649c11015650, scope.project.id=3b402af2708b41df849963f81e65797e, user.id=4585768734fc4332a61c845383c1482c) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:39.051802 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a0335-022e-4aa7-96ea-534c9d5903e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:39.053615 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a0335-022e-4aa7-96ea-534c9d5903e9 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:39.062537 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 83/159] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:53:39.064959 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 77/160] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:53:38 2022] GET /identity/v3/role_assignments?user.id=4585768734fc4332a61c845383c1482c&scope.project.id=3b402af2708b41df849963f81e65797e&role.id=c9da950c75da4939b8bd649c11015650 => generated 608 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:39.408702 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437ac6f-3db8-410c-acbf-60e7334f5a4d None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:39.446603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:39.530274 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437ac6f-3db8-410c-acbf-60e7334f5a4d None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:39.574199 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eac48a-dc7b-4c9b-a77d-507e4cdd0c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:39.574876 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eac48a-dc7b-4c9b-a77d-507e4cdd0c56 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:39.575158 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eac48a-dc7b-4c9b-a77d-507e4cdd0c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:39.575429 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eac48a-dc7b-4c9b-a77d-507e4cdd0c56 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:39.586346 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eac48a-dc7b-4c9b-a77d-507e4cdd0c56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:53:39.591318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-91eac48a-dc7b-4c9b-a77d-507e4cdd0c56 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:53:39.591318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:53:39.591318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:53:39.591318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:53:39.591318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:53:39.591318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:53:39.591318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:53:39.591318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:53:39.591318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:53:39.591318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:53:39.591318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:53:39.591318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:53:39.591318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:53:39.591318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:53:39.591318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:53:39.591318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:53:39.592247 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:53:39.592247 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:53:39.592247 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:53:39.592247 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:53:39.592247 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:53:39.592247 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:53:39.592247 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:53:39.592247 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:53:39.592247 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:53:39.592247 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:53:39.592247 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:53:39.592247 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:53:39.592247 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:53:39.592247 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:53:39.592247 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:53:39.592247 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:53:39.593389 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:53:39.593389 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:53:39.593389 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:53:39.593389 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:53:39.593389 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:53:39.593389 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:53:39.593389 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:53:39.593389 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:53:39.593389 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:53:39.593389 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:53:39.593389 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:53:39.593389 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:53:39.593389 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 84/161] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:53:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:53:39.599288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a0335-022e-4aa7-96ea-534c9d5903e9 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:39.671979 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eac48a-dc7b-4c9b-a77d-507e4cdd0c56 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:39.754903 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a0335-022e-4aa7-96ea-534c9d5903e9 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:39.764764 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:39.815569 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:39.825439 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf62f6-bcb2-4df4-a045-8ec2d4d75be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:39.826087 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf62f6-bcb2-4df4-a045-8ec2d4d75be3 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:39.826367 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf62f6-bcb2-4df4-a045-8ec2d4d75be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:39.826638 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf62f6-bcb2-4df4-a045-8ec2d4d75be3 None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:39.827414 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf62f6-bcb2-4df4-a045-8ec2d4d75be3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:39.827720 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf62f6-bcb2-4df4-a045-8ec2d4d75be3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:39.829766 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf62f6-bcb2-4df4-a045-8ec2d4d75be3 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:39.849219 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 78/162] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 14:53:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:39.853063 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eac48a-dc7b-4c9b-a77d-507e4cdd0c56 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:39.854953 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf62f6-bcb2-4df4-a045-8ec2d4d75be3 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:39.929035 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf147439-6c59-4835-aa75-156cd9151294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:39.929593 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf147439-6c59-4835-aa75-156cd9151294 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:39.929866 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf147439-6c59-4835-aa75-156cd9151294 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:39.930142 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf147439-6c59-4835-aa75-156cd9151294 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:39.956929 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf147439-6c59-4835-aa75-156cd9151294 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:53:39.961889 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-cf147439-6c59-4835-aa75-156cd9151294 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:53:39.961889 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:53:39.961889 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:53:39.961889 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:53:39.961889 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:53:39.961889 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:53:39.961889 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:53:39.961889 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:53:39.961889 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:53:39.961889 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:53:39.961889 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:53:39.961889 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:53:39.961889 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:53:39.961889 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:53:39.961889 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:53:39.961889 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:53:39.962762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:53:39.962762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:53:39.962762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:53:39.962762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:53:39.962762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:53:39.962762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:53:39.962762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:53:39.962762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:53:39.962762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:53:39.962762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:53:39.962762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:53:39.962762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:53:39.962762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:53:39.962762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:53:39.962762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:53:39.962762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:53:39.963960 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:53:39.963960 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:53:39.963960 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:53:39.963960 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:53:39.963960 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:53:39.963960 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:53:39.963960 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:53:39.963960 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:53:39.963960 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:53:39.963960 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:53:39.963960 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:53:39.963960 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:53:39.963960 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 85/163] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 14:53:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:53:39.968966 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf147439-6c59-4835-aa75-156cd9151294 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:40.043028 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf62f6-bcb2-4df4-a045-8ec2d4d75be3 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:40.084088 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5259513-774a-4b9f-abac-c9023d5492d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:40.085401 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5259513-774a-4b9f-abac-c9023d5492d6 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:40.087007 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5259513-774a-4b9f-abac-c9023d5492d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:40.087697 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5259513-774a-4b9f-abac-c9023d5492d6 None admin] PATH_INFO: `/v3/users/4585768734fc4332a61c845383c1482c` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:40.089857 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5259513-774a-4b9f-abac-c9023d5492d6 None admin] RBAC: Authorizing `identity:get_user(user_id=4585768734fc4332a61c845383c1482c)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:40.093008 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5259513-774a-4b9f-abac-c9023d5492d6 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:40.096614 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 79/164] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:53:39 2022] GET /identity/v3/users/4585768734fc4332a61c845383c1482c => generated 275 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:53:40.098921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca62618-4b78-401d-901d-88a1771d5ac3 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:40.101842 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca62618-4b78-401d-901d-88a1771d5ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:40.104927 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca62618-4b78-401d-901d-88a1771d5ac3 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:40.106988 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 80/165] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:40.108823 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca62618-4b78-401d-901d-88a1771d5ac3 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:40.183813 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:40.207935 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf147439-6c59-4835-aa75-156cd9151294 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:40.272911 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3df455-0b64-4447-a4bf-52f0b065ba6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:40.273535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3df455-0b64-4447-a4bf-52f0b065ba6a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:40.273816 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3df455-0b64-4447-a4bf-52f0b065ba6a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:40.274091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3df455-0b64-4447-a4bf-52f0b065ba6a None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:40.274772 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3df455-0b64-4447-a4bf-52f0b065ba6a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:40.275092 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3df455-0b64-4447-a4bf-52f0b065ba6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:40.276508 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3df455-0b64-4447-a4bf-52f0b065ba6a None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:40.284322 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca62618-4b78-401d-901d-88a1771d5ac3 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:40.285938 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 86/166] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:53:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:40.291311 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1f543-5171-485e-9b87-19550f67a14d None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:40.291609 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1f543-5171-485e-9b87-19550f67a14d None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:40.291890 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1f543-5171-485e-9b87-19550f67a14d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:40.316134 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:40.332193 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-d3d1f543-5171-485e-9b87-19550f67a14d None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:40.333405 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-d3d1f543-5171-485e-9b87-19550f67a14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:40.360971 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bbf2d1-f799-47ae-b448-01b185a1aa47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:40.361589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbf2d1-f799-47ae-b448-01b185a1aa47 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:40.361870 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbf2d1-f799-47ae-b448-01b185a1aa47 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:40.362144 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbf2d1-f799-47ae-b448-01b185a1aa47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:40.472210 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 87/167] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:40.473084 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d1f543-5171-485e-9b87-19550f67a14d None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:40.507848 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bbf2d1-f799-47ae-b448-01b185a1aa47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:40.509507 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bbf2d1-f799-47ae-b448-01b185a1aa47 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:40.590041 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:40.669371 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d1f543-5171-485e-9b87-19550f67a14d None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:40.734787 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 81/168] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:53:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 626 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:53:40.739117 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f262a8-0121-42a8-ac81-5c79d28ebbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:40.739805 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f262a8-0121-42a8-ac81-5c79d28ebbc5 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:40.740055 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f262a8-0121-42a8-ac81-5c79d28ebbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:40.740329 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f262a8-0121-42a8-ac81-5c79d28ebbc5 None admin] PATH_INFO: `/v3/users/4585768734fc4332a61c845383c1482c` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:40.741076 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bbf2d1-f799-47ae-b448-01b185a1aa47 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:40.741505 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f262a8-0121-42a8-ac81-5c79d28ebbc5 None admin] RBAC: Authorizing `identity:get_user(user_id=4585768734fc4332a61c845383c1482c)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:40.743074 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f262a8-0121-42a8-ac81-5c79d28ebbc5 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:40.744796 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 88/169] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:53:40 2022] GET /identity/v3/users/4585768734fc4332a61c845383c1482c => generated 275 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:40.756668 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f262a8-0121-42a8-ac81-5c79d28ebbc5 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:40.768680 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:40.893606 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f262a8-0121-42a8-ac81-5c79d28ebbc5 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:40.963574 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6235b-4e57-4d5a-b8c0-2ef38b8fd9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:40.965371 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6235b-4e57-4d5a-b8c0-2ef38b8fd9d5 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:40.967337 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6235b-4e57-4d5a-b8c0-2ef38b8fd9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:40.967847 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6235b-4e57-4d5a-b8c0-2ef38b8fd9d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:40.997314 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bbf2d1-f799-47ae-b448-01b185a1aa47 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:41.054335 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd36228-ce96-4bd0-ad1c-eb6ed71c9b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:41.055023 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd36228-ce96-4bd0-ad1c-eb6ed71c9b41 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:41.055304 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd36228-ce96-4bd0-ad1c-eb6ed71c9b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:41.055577 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd36228-ce96-4bd0-ad1c-eb6ed71c9b41 None admin] PATH_INFO: `/v3/projects/3dc3bfe113c841f098a02b2b1ec28edc` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:41.056774 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd36228-ce96-4bd0-ad1c-eb6ed71c9b41 None admin] RBAC: Authorizing `identity:get_project(project_id=3dc3bfe113c841f098a02b2b1ec28edc)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:41.058506 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd36228-ce96-4bd0-ad1c-eb6ed71c9b41 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:41.067163 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 82/170] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:53:40 2022] GET /identity/v3/projects/3dc3bfe113c841f098a02b2b1ec28edc => generated 312 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:53:41.069328 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee0e4d-800d-4558-bb7e-2f4d3c74531e None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:41.071152 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee0e4d-800d-4558-bb7e-2f4d3c74531e None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:41.072546 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee0e4d-800d-4558-bb7e-2f4d3c74531e None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:41.081846 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 83/171] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:41 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:41.082950 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee0e4d-800d-4558-bb7e-2f4d3c74531e None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:41.090085 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a6235b-4e57-4d5a-b8c0-2ef38b8fd9d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:41.092964 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a6235b-4e57-4d5a-b8c0-2ef38b8fd9d5 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:41.103158 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:41.116529 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:41.274063 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee0e4d-800d-4558-bb7e-2f4d3c74531e None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:41.291910 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 89/172] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:53:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 14:53:41.292845 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6235b-4e57-4d5a-b8c0-2ef38b8fd9d5 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:41.326260 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e7cd7d-2fa6-4b38-8a22-b6da1000f58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:41.326945 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7cd7d-2fa6-4b38-8a22-b6da1000f58a None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:41.327227 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7cd7d-2fa6-4b38-8a22-b6da1000f58a None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:41.327494 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7cd7d-2fa6-4b38-8a22-b6da1000f58a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:41.338423 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e7cd7d-2fa6-4b38-8a22-b6da1000f58a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:53:41.343614 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-a3e7cd7d-2fa6-4b38-8a22-b6da1000f58a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:53:41.343614 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:53:41.343614 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:53:41.343614 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:53:41.343614 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:53:41.343614 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:53:41.343614 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:53:41.343614 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:53:41.343614 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:53:41.343614 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:53:41.343614 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:53:41.343614 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:53:41.343614 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:53:41.343614 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:53:41.343614 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:53:41.343614 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:53:41.344693 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:53:41.344693 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:53:41.344693 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:53:41.344693 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:53:41.344693 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:53:41.344693 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:53:41.344693 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:53:41.344693 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:53:41.344693 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:53:41.344693 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:53:41.344693 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:53:41.344693 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:53:41.344693 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:53:41.344693 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:53:41.344693 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:53:41.344693 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:53:41.345550 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:53:41.345550 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:53:41.345550 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:53:41.345550 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:53:41.345550 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:53:41.345550 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:53:41.345550 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:53:41.345550 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:53:41.345550 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:53:41.345550 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:53:41.345550 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:53:41.345550 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:53:41.345550 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 84/173] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:53:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:53:41.346214 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84dbe59-0a6b-4884-b0fb-7528175ad905 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:41.346777 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84dbe59-0a6b-4884-b0fb-7528175ad905 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:41.346777 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84dbe59-0a6b-4884-b0fb-7528175ad905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:41.374398 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-c84dbe59-0a6b-4884-b0fb-7528175ad905 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:41.375854 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-c84dbe59-0a6b-4884-b0fb-7528175ad905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:41.451722 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 85/174] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:41.454330 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84dbe59-0a6b-4884-b0fb-7528175ad905 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:41.471202 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:41.505654 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6235b-4e57-4d5a-b8c0-2ef38b8fd9d5 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:41.526908 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:41.543542 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b7902-55b4-42e5-a82e-6f80e307cd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:41.544578 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b7902-55b4-42e5-a82e-6f80e307cd83 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:41.545157 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b7902-55b4-42e5-a82e-6f80e307cd83 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:41.545542 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b7902-55b4-42e5-a82e-6f80e307cd83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:41.546794 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b7902-55b4-42e5-a82e-6f80e307cd83 None admin] RBAC: Adding query filter params (role.id=c9da950c75da4939b8bd649c11015650, scope.project.id=3dc3bfe113c841f098a02b2b1ec28edc, user.id=4585768734fc4332a61c845383c1482c) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:41.547249 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b7902-55b4-42e5-a82e-6f80e307cd83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:41.549961 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b7902-55b4-42e5-a82e-6f80e307cd83 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:41.557127 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 90/175] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:53:41 2022] GET /identity/v3/role_assignments?user.id=4585768734fc4332a61c845383c1482c&scope.project.id=3dc3bfe113c841f098a02b2b1ec28edc&role.id=c9da950c75da4939b8bd649c11015650 => generated 608 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:41.558160 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b7902-55b4-42e5-a82e-6f80e307cd83 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:41.679849 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b7902-55b4-42e5-a82e-6f80e307cd83 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:41.693164 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84dbe59-0a6b-4884-b0fb-7528175ad905 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:41.718236 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139e8a9-4813-48ce-bac8-d005457d7a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:41.719761 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139e8a9-4813-48ce-bac8-d005457d7a3e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:41.720189 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139e8a9-4813-48ce-bac8-d005457d7a3e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:41.720570 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139e8a9-4813-48ce-bac8-d005457d7a3e None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:41.721394 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d139e8a9-4813-48ce-bac8-d005457d7a3e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:41.721762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d139e8a9-4813-48ce-bac8-d005457d7a3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:41.723502 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d139e8a9-4813-48ce-bac8-d005457d7a3e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:41.733526 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 91/176] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:53:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:41.734880 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87ce3a-5d9f-4a61-b00e-fde175c783ea None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:41.734880 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87ce3a-5d9f-4a61-b00e-fde175c783ea None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:41.735190 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87ce3a-5d9f-4a61-b00e-fde175c783ea None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:41.736610 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 92/177] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:41.737826 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b356288-8906-4a19-b9b6-2f1606536a85 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:41.738225 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b356288-8906-4a19-b9b6-2f1606536a85 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:41.738600 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b356288-8906-4a19-b9b6-2f1606536a85 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:41.740989 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 93/178] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:41.746485 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e4853-e7f9-4afd-85e3-29972afea4d2 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:41.746924 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e4853-e7f9-4afd-85e3-29972afea4d2 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:41.747312 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e4853-e7f9-4afd-85e3-29972afea4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:41.766537 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b15a5-8ea3-44eb-aba8-11bac4a538c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:41.767343 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b15a5-8ea3-44eb-aba8-11bac4a538c0 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:41.767668 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b15a5-8ea3-44eb-aba8-11bac4a538c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:41.767942 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b15a5-8ea3-44eb-aba8-11bac4a538c0 None admin] PATH_INFO: `/v3/projects/3b402af2708b41df849963f81e65797e` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:41.769326 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-0e1e4853-e7f9-4afd-85e3-29972afea4d2 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:41.770380 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442b15a5-8ea3-44eb-aba8-11bac4a538c0 None admin] RBAC: Authorizing `identity:get_project(project_id=3b402af2708b41df849963f81e65797e)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:41.770753 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-0e1e4853-e7f9-4afd-85e3-29972afea4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:41.772026 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442b15a5-8ea3-44eb-aba8-11bac4a538c0 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:41.774275 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 86/179] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:53:41 2022] GET /identity/v3/projects/3b402af2708b41df849963f81e65797e => generated 298 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:41.775182 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b15a5-8ea3-44eb-aba8-11bac4a538c0 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:41.799078 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:41.852344 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 94/180] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:41.853820 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805aef7-6ce8-4c37-9d21-068fa0c63b3a None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:41.854160 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805aef7-6ce8-4c37-9d21-068fa0c63b3a None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:41.854542 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805aef7-6ce8-4c37-9d21-068fa0c63b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:41.855738 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:41.878085 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-b805aef7-6ce8-4c37-9d21-068fa0c63b3a None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:41.881334 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-b805aef7-6ce8-4c37-9d21-068fa0c63b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:41.912387 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b15a5-8ea3-44eb-aba8-11bac4a538c0 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:41.956087 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c37e19e-6767-49ba-8aba-9e697de89fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:41.957456 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37e19e-6767-49ba-8aba-9e697de89fcf None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:41.957887 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37e19e-6767-49ba-8aba-9e697de89fcf None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:41.958580 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37e19e-6767-49ba-8aba-9e697de89fcf None admin] PATH_INFO: `/v3/users/53878dbb519c4b3d8858375928f1c2d4` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:41.960255 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c37e19e-6767-49ba-8aba-9e697de89fcf None admin] RBAC: Authorizing `identity:get_user(user_id=53878dbb519c4b3d8858375928f1c2d4)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:41.962647 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c37e19e-6767-49ba-8aba-9e697de89fcf None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:41.965114 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 87/181] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:53:41 2022] GET /identity/v3/users/53878dbb519c4b3d8858375928f1c2d4 => generated 247 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:41.966031 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c37e19e-6767-49ba-8aba-9e697de89fcf None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:41.993966 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 95/182] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:41.996571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b805aef7-6ce8-4c37-9d21-068fa0c63b3a None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:42.151213 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c37e19e-6767-49ba-8aba-9e697de89fcf None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:42.183385 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:42.190013 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:42.195049 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9bc983-3dc4-45c2-8508-8fccfdc305bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:42.195669 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bc983-3dc4-45c2-8508-8fccfdc305bc None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:42.195954 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bc983-3dc4-45c2-8508-8fccfdc305bc None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:42.196228 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bc983-3dc4-45c2-8508-8fccfdc305bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:42.196961 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9bc983-3dc4-45c2-8508-8fccfdc305bc None admin] RBAC: Adding query filter params (role.id=d1818eb7f2bd467fbf75bad5cc8e0aea, scope.project.id=3b402af2708b41df849963f81e65797e, user.id=4585768734fc4332a61c845383c1482c) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:42.197269 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9bc983-3dc4-45c2-8508-8fccfdc305bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:42.199510 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9bc983-3dc4-45c2-8508-8fccfdc305bc None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:42.201209 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b805aef7-6ce8-4c37-9d21-068fa0c63b3a None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:42.206746 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 88/183] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:53:41 2022] GET /identity/v3/role_assignments?user.id=4585768734fc4332a61c845383c1482c&scope.project.id=3b402af2708b41df849963f81e65797e&role.id=d1818eb7f2bd467fbf75bad5cc8e0aea => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:42.207423 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9bc983-3dc4-45c2-8508-8fccfdc305bc None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:42.248608 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be8a22-b99f-44b6-bcb0-021c6960db9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:42.249272 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be8a22-b99f-44b6-bcb0-021c6960db9a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:42.249558 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be8a22-b99f-44b6-bcb0-021c6960db9a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:42.249828 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be8a22-b99f-44b6-bcb0-021c6960db9a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:42.267930 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be8a22-b99f-44b6-bcb0-021c6960db9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:53:42.276577 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-80be8a22-b99f-44b6-bcb0-021c6960db9a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:53:42.276577 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:53:42.276577 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:53:42.276577 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:53:42.276577 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:53:42.276577 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:53:42.276577 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:53:42.276577 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:53:42.276577 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:53:42.276577 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:53:42.276577 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:53:42.276577 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:53:42.276577 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:53:42.276577 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:53:42.276577 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:53:42.276577 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:53:42.277580 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:53:42.277580 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:53:42.277580 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:53:42.277580 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:53:42.277580 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:53:42.277580 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:53:42.277580 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:53:42.277580 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:53:42.277580 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:53:42.277580 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:53:42.277580 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:53:42.277580 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:53:42.277580 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:53:42.277580 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:53:42.277580 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:53:42.277580 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:53:42.278390 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:53:42.278390 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:53:42.278390 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:53:42.278390 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:53:42.278390 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:53:42.278390 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:53:42.278390 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:53:42.278390 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:53:42.278390 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:53:42.278390 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:53:42.278390 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:53:42.278390 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:53:42.278390 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 96/184] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:53:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:53:42.279581 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1317ab03-4fc5-41f4-a0dd-45df193750f2 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:42.279581 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1317ab03-4fc5-41f4-a0dd-45df193750f2 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:42.279851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1317ab03-4fc5-41f4-a0dd-45df193750f2 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:42.281109 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 97/185] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:42.281897 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1317ab03-4fc5-41f4-a0dd-45df193750f2 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:42.349630 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9bc983-3dc4-45c2-8508-8fccfdc305bc None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:42.388262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d92332-dfee-4c93-885d-7bea0aa905e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:42.389242 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d92332-dfee-4c93-885d-7bea0aa905e6 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:42.390332 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d92332-dfee-4c93-885d-7bea0aa905e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:42.390332 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d92332-dfee-4c93-885d-7bea0aa905e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:42.399778 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1317ab03-4fc5-41f4-a0dd-45df193750f2 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:42.443250 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd2e14-e251-462f-b920-a0abbcec9449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:42.443821 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd2e14-e251-462f-b920-a0abbcec9449 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:42.444462 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd2e14-e251-462f-b920-a0abbcec9449 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:42.444741 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd2e14-e251-462f-b920-a0abbcec9449 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:42.457485 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:42.460770 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd2e14-e251-462f-b920-a0abbcec9449 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:53:42.465928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-f7dd2e14-e251-462f-b920-a0abbcec9449 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:53:42.465928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:53:42.465928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:53:42.465928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:53:42.465928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:53:42.465928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:53:42.465928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:53:42.465928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:53:42.465928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:53:42.465928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:53:42.465928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:53:42.465928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:53:42.465928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:53:42.465928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:53:42.465928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:53:42.465928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:53:42.466969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:53:42.466969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:53:42.466969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:53:42.466969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:53:42.466969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:53:42.466969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:53:42.466969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:53:42.466969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:53:42.466969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:53:42.466969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:53:42.466969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:53:42.466969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:53:42.466969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:53:42.466969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:53:42.466969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:53:42.466969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:53:42.467850 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:53:42.467850 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:53:42.467850 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:53:42.467850 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:53:42.467850 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:53:42.467850 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:53:42.467850 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:53:42.467850 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:53:42.467850 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:53:42.467850 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:53:42.467850 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:53:42.467850 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:53:42.467850 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 98/186] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 14:53:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:53:42.468544 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd2e14-e251-462f-b920-a0abbcec9449 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:42.470850 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d92332-dfee-4c93-885d-7bea0aa905e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:42.471868 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d92332-dfee-4c93-885d-7bea0aa905e6 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:42.484445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:42.571111 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 89/187] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:53:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:53:42.572657 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99d347-3bb3-4665-9134-30ed50d18423 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:42.572946 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99d347-3bb3-4665-9134-30ed50d18423 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:42.573218 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99d347-3bb3-4665-9134-30ed50d18423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:42.578827 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd2e14-e251-462f-b920-a0abbcec9449 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:42.591275 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-2b99d347-3bb3-4665-9134-30ed50d18423 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:42.592462 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-2b99d347-3bb3-4665-9134-30ed50d18423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:42.615778 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043db365-34ed-4a69-a6d5-7aede53b006b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:42.616389 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043db365-34ed-4a69-a6d5-7aede53b006b None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:42.616669 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043db365-34ed-4a69-a6d5-7aede53b006b None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:42.616943 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043db365-34ed-4a69-a6d5-7aede53b006b None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:42.617605 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043db365-34ed-4a69-a6d5-7aede53b006b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:42.617904 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043db365-34ed-4a69-a6d5-7aede53b006b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:42.619339 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043db365-34ed-4a69-a6d5-7aede53b006b None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:42.627328 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 99/188] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:53:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:42.628212 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043db365-34ed-4a69-a6d5-7aede53b006b None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:42.665385 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 90/189] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:42.666230 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b99d347-3bb3-4665-9134-30ed50d18423 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:42.700667 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:42.701119 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:42.739106 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043db365-34ed-4a69-a6d5-7aede53b006b None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:42.774487 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e733b-ffb1-4377-bd7f-e6e59fe8f8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:42.775152 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e733b-ffb1-4377-bd7f-e6e59fe8f8bd None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:42.775435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e733b-ffb1-4377-bd7f-e6e59fe8f8bd None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:42.775708 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e733b-ffb1-4377-bd7f-e6e59fe8f8bd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:42.776823 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275e733b-ffb1-4377-bd7f-e6e59fe8f8bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:42.778317 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275e733b-ffb1-4377-bd7f-e6e59fe8f8bd None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:42.780095 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 100/190] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:53:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:42.780773 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e733b-ffb1-4377-bd7f-e6e59fe8f8bd None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:42.796267 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b99d347-3bb3-4665-9134-30ed50d18423 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:42.834161 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3115d97d-fe89-4f7a-9a32-4d5f2df26a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:42.834773 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115d97d-fe89-4f7a-9a32-4d5f2df26a30 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:42.835083 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115d97d-fe89-4f7a-9a32-4d5f2df26a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:42.835352 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115d97d-fe89-4f7a-9a32-4d5f2df26a30 None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:42.836009 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3115d97d-fe89-4f7a-9a32-4d5f2df26a30 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:42.836310 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3115d97d-fe89-4f7a-9a32-4d5f2df26a30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:42.837703 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3115d97d-fe89-4f7a-9a32-4d5f2df26a30 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:42.846651 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 91/191] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:53:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:42.852322 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3115d97d-fe89-4f7a-9a32-4d5f2df26a30 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:42.892333 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e733b-ffb1-4377-bd7f-e6e59fe8f8bd None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:42.932327 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0788744-9838-4962-84b3-c5164f8772d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:42.933572 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0788744-9838-4962-84b3-c5164f8772d8 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:42.933921 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0788744-9838-4962-84b3-c5164f8772d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:42.934202 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0788744-9838-4962-84b3-c5164f8772d8 None admin] PATH_INFO: `/v3/projects/3b402af2708b41df849963f81e65797e` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:42.935447 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0788744-9838-4962-84b3-c5164f8772d8 None admin] RBAC: Authorizing `identity:get_project(project_id=3b402af2708b41df849963f81e65797e)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:42.936905 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0788744-9838-4962-84b3-c5164f8772d8 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:42.939131 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 101/192] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:53:42 2022] GET /identity/v3/projects/3b402af2708b41df849963f81e65797e => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:42.939933 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0788744-9838-4962-84b3-c5164f8772d8 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:42.944854 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:42.951041 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:42.961687 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3115d97d-fe89-4f7a-9a32-4d5f2df26a30 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:42.997643 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c774b-e1ac-4958-9887-93c9a6cba91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:42.998284 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c774b-e1ac-4958-9887-93c9a6cba91d None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:42.998567 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c774b-e1ac-4958-9887-93c9a6cba91d None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:42.998864 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c774b-e1ac-4958-9887-93c9a6cba91d None admin] PATH_INFO: `/v3/users/91b926a36a5f4024a2cd0788d2867305` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:43.013685 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059c774b-e1ac-4958-9887-93c9a6cba91d None admin] RBAC: Authorizing `identity:get_user(user_id=91b926a36a5f4024a2cd0788d2867305)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:43.015168 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059c774b-e1ac-4958-9887-93c9a6cba91d None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:43.017347 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 92/193] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:53:42 2022] GET /identity/v3/users/91b926a36a5f4024a2cd0788d2867305 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:43.018111 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c774b-e1ac-4958-9887-93c9a6cba91d None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:43.042463 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0788744-9838-4962-84b3-c5164f8772d8 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:43.076937 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d671203-744b-4ea5-94de-cb70ad9a538c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:43.077546 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d671203-744b-4ea5-94de-cb70ad9a538c None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:43.077825 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d671203-744b-4ea5-94de-cb70ad9a538c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:43.078098 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d671203-744b-4ea5-94de-cb70ad9a538c None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:43.079604 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d671203-744b-4ea5-94de-cb70ad9a538c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:43.081144 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d671203-744b-4ea5-94de-cb70ad9a538c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:43.101990 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 102/194] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 14:53:42 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:53:43.102827 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d671203-744b-4ea5-94de-cb70ad9a538c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:43.138502 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c774b-e1ac-4958-9887-93c9a6cba91d None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:43.180008 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e68d1a-ebfa-45aa-8d25-9435eaddaba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:43.180961 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e68d1a-ebfa-45aa-8d25-9435eaddaba3 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:43.181407 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e68d1a-ebfa-45aa-8d25-9435eaddaba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:43.181683 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e68d1a-ebfa-45aa-8d25-9435eaddaba3 None admin] PATH_INFO: `/v3/users/91b926a36a5f4024a2cd0788d2867305` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:43.182504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:43.183229 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e68d1a-ebfa-45aa-8d25-9435eaddaba3 None admin] RBAC: Authorizing `identity:get_user(user_id=91b926a36a5f4024a2cd0788d2867305)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:43.184798 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e68d1a-ebfa-45aa-8d25-9435eaddaba3 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:43.191915 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 93/195] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:53:43 2022] GET /identity/v3/users/91b926a36a5f4024a2cd0788d2867305 => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:43.191915 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e68d1a-ebfa-45aa-8d25-9435eaddaba3 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:43.203533 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:43.236748 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d671203-744b-4ea5-94de-cb70ad9a538c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:43.274614 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2686fb48-1562-4805-94f6-e12b2a525957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:43.275137 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686fb48-1562-4805-94f6-e12b2a525957 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:43.276192 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686fb48-1562-4805-94f6-e12b2a525957 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:43.276192 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686fb48-1562-4805-94f6-e12b2a525957 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:43.290123 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2686fb48-1562-4805-94f6-e12b2a525957 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:53:43.295606 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-2686fb48-1562-4805-94f6-e12b2a525957 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:53:43.295606 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:53:43.295606 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:53:43.295606 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:53:43.295606 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:53:43.295606 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:53:43.295606 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:53:43.295606 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:53:43.295606 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:53:43.295606 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:53:43.295606 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:53:43.295606 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:53:43.295606 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:53:43.295606 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:53:43.295606 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:53:43.295606 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:53:43.296491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:53:43.296491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:53:43.296491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:53:43.296491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:53:43.296491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:53:43.296491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:53:43.296491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:53:43.296491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:53:43.296491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:53:43.296491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:53:43.296491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:53:43.296491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:53:43.296491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:53:43.296491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:53:43.296491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:53:43.296491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:53:43.297334 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:53:43.297334 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:53:43.297334 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:53:43.297334 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:53:43.297334 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:53:43.297334 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:53:43.297334 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:53:43.297334 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:53:43.297334 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:53:43.297334 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:53:43.297334 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:53:43.297334 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:53:43.297334 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 103/196] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:53:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:53:43.298243 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2686fb48-1562-4805-94f6-e12b2a525957 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:43.312492 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e68d1a-ebfa-45aa-8d25-9435eaddaba3 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:43.352156 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b3e82e-616c-4f3c-9079-3452df0d5b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:43.352764 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3e82e-616c-4f3c-9079-3452df0d5b28 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:43.353040 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3e82e-616c-4f3c-9079-3452df0d5b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:43.353307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3e82e-616c-4f3c-9079-3452df0d5b28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:43.354678 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b3e82e-616c-4f3c-9079-3452df0d5b28 None admin] RBAC: Adding query filter params (role.id=99e8fe7df8e7446680a81d4eaaa1cb68, scope.project.id=3b402af2708b41df849963f81e65797e, user.id=53878dbb519c4b3d8858375928f1c2d4) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:43.354678 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b3e82e-616c-4f3c-9079-3452df0d5b28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:43.355797 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b3e82e-616c-4f3c-9079-3452df0d5b28 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:43.367127 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 94/197] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:53:43 2022] GET /identity/v3/role_assignments?user.id=53878dbb519c4b3d8858375928f1c2d4&scope.project.id=3b402af2708b41df849963f81e65797e&role.id=99e8fe7df8e7446680a81d4eaaa1cb68 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:43.367127 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b3e82e-616c-4f3c-9079-3452df0d5b28 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:43.428906 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2686fb48-1562-4805-94f6-e12b2a525957 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:43.470444 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:43.473312 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:43.488686 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e5a49-2cf6-4db7-85cd-edf9a199465d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:43.489331 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e5a49-2cf6-4db7-85cd-edf9a199465d None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:43.489611 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e5a49-2cf6-4db7-85cd-edf9a199465d None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:43.489882 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e5a49-2cf6-4db7-85cd-edf9a199465d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:43.508502 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b3e82e-616c-4f3c-9079-3452df0d5b28 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:43.550086 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04d4aa-3fcd-4018-a41e-f937d483c2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:43.550943 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04d4aa-3fcd-4018-a41e-f937d483c2cd None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:43.551618 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04d4aa-3fcd-4018-a41e-f937d483c2cd None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:43.551618 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04d4aa-3fcd-4018-a41e-f937d483c2cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:43.587453 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e5a49-2cf6-4db7-85cd-edf9a199465d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:43.589515 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e5a49-2cf6-4db7-85cd-edf9a199465d None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:43.634855 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04d4aa-3fcd-4018-a41e-f937d483c2cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:43.636888 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04d4aa-3fcd-4018-a41e-f937d483c2cd None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:43.710204 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 104/198] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:53:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:53:43.711401 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e5a49-2cf6-4db7-85cd-edf9a199465d None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:43.724174 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:43.744127 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:43.757760 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 95/199] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:53:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:53:43.758939 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04d4aa-3fcd-4018-a41e-f937d483c2cd None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:43.864101 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e5a49-2cf6-4db7-85cd-edf9a199465d None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:43.883562 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04d4aa-3fcd-4018-a41e-f937d483c2cd None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:43.918105 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1d492-40e3-4604-9b5d-45dd944ad7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:43.918105 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1d492-40e3-4604-9b5d-45dd944ad7c5 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:43.918105 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1d492-40e3-4604-9b5d-45dd944ad7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:43.918485 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1d492-40e3-4604-9b5d-45dd944ad7c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:43.918976 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f1d492-40e3-4604-9b5d-45dd944ad7c5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:43.919413 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f1d492-40e3-4604-9b5d-45dd944ad7c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:43.922520 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f1d492-40e3-4604-9b5d-45dd944ad7c5 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:43.938839 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 105/200] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 14:53:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:43.938839 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1d492-40e3-4604-9b5d-45dd944ad7c5 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:43.943065 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a03f3d-66bc-42d6-ab78-ff459499881b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:43.943885 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a03f3d-66bc-42d6-ab78-ff459499881b None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:43.944317 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a03f3d-66bc-42d6-ab78-ff459499881b None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:43.944724 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a03f3d-66bc-42d6-ab78-ff459499881b None admin] PATH_INFO: `/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:43.946015 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a03f3d-66bc-42d6-ab78-ff459499881b None admin] RBAC: Authorizing `identity:get_project(project_id=8f8cdaafdd52411bb20efe51ddc2565c)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:43.948134 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a03f3d-66bc-42d6-ab78-ff459499881b None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:43.950361 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 96/201] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:53:43 2022] GET /identity/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:43.951612 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a03f3d-66bc-42d6-ab78-ff459499881b None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:44.006727 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:44.054198 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:44.077407 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1d492-40e3-4604-9b5d-45dd944ad7c5 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:44.082392 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a03f3d-66bc-42d6-ab78-ff459499881b None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:44.113653 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2983ab9d-0144-41c7-b80d-0d2e0ad16fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:44.114559 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983ab9d-0144-41c7-b80d-0d2e0ad16fcd None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:44.115070 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983ab9d-0144-41c7-b80d-0d2e0ad16fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:44.115515 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983ab9d-0144-41c7-b80d-0d2e0ad16fcd None admin] PATH_INFO: `/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:44.116809 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2983ab9d-0144-41c7-b80d-0d2e0ad16fcd None admin] RBAC: Authorizing `identity:get_project(project_id=8f8cdaafdd52411bb20efe51ddc2565c)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:44.118465 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2983ab9d-0144-41c7-b80d-0d2e0ad16fcd None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:44.120428 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 106/202] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:53:43 2022] GET /identity/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:44.121398 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2983ab9d-0144-41c7-b80d-0d2e0ad16fcd None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:44.128207 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa66c2e9-b8ba-4d99-9f47-ada4fbbbbec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:44.128858 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66c2e9-b8ba-4d99-9f47-ada4fbbbbec7 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:44.129133 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66c2e9-b8ba-4d99-9f47-ada4fbbbbec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:44.129401 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66c2e9-b8ba-4d99-9f47-ada4fbbbbec7 None admin] PATH_INFO: `/v3/users/ed513a0aabf84899951cac87303e0a8f` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:44.130562 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa66c2e9-b8ba-4d99-9f47-ada4fbbbbec7 None admin] RBAC: Authorizing `identity:get_user(user_id=ed513a0aabf84899951cac87303e0a8f)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:44.132611 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa66c2e9-b8ba-4d99-9f47-ada4fbbbbec7 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:44.133933 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 97/203] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:53:43 2022] GET /identity/v3/users/ed513a0aabf84899951cac87303e0a8f => generated 289 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:44.135066 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa66c2e9-b8ba-4d99-9f47-ada4fbbbbec7 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:44.226384 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2983ab9d-0144-41c7-b80d-0d2e0ad16fcd None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:44.246763 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa66c2e9-b8ba-4d99-9f47-ada4fbbbbec7 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:44.258445 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:44.263242 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0fd3e-dac1-4537-a080-cbbd4931ee40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:44.264606 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0fd3e-dac1-4537-a080-cbbd4931ee40 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:44.265388 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0fd3e-dac1-4537-a080-cbbd4931ee40 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:44.266721 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0fd3e-dac1-4537-a080-cbbd4931ee40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:44.268505 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd0fd3e-dac1-4537-a080-cbbd4931ee40 None admin] RBAC: Adding query filter params (role.id=99e8fe7df8e7446680a81d4eaaa1cb68, scope.project.id=8f8cdaafdd52411bb20efe51ddc2565c, user.id=91b926a36a5f4024a2cd0788d2867305) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:44.269340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd0fd3e-dac1-4537-a080-cbbd4931ee40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:44.272473 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd0fd3e-dac1-4537-a080-cbbd4931ee40 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:44.281004 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 107/204] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:53:44 2022] GET /identity/v3/role_assignments?user.id=91b926a36a5f4024a2cd0788d2867305&scope.project.id=8f8cdaafdd52411bb20efe51ddc2565c&role.id=99e8fe7df8e7446680a81d4eaaa1cb68 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:44.283653 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0fd3e-dac1-4537-a080-cbbd4931ee40 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:44.287968 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b4228-f486-4830-9c91-97ad6a857a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:44.288732 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b4228-f486-4830-9c91-97ad6a857a47 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:44.288867 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b4228-f486-4830-9c91-97ad6a857a47 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:44.289117 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b4228-f486-4830-9c91-97ad6a857a47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:44.289947 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941b4228-f486-4830-9c91-97ad6a857a47 None admin] RBAC: Adding query filter params (role.id=d1818eb7f2bd467fbf75bad5cc8e0aea, scope.project.id=8f8cdaafdd52411bb20efe51ddc2565c, user.id=91b926a36a5f4024a2cd0788d2867305) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:44.290260 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941b4228-f486-4830-9c91-97ad6a857a47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:44.291801 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941b4228-f486-4830-9c91-97ad6a857a47 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:44.299250 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 98/205] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:53:44 2022] GET /identity/v3/role_assignments?user.id=91b926a36a5f4024a2cd0788d2867305&scope.project.id=8f8cdaafdd52411bb20efe51ddc2565c&role.id=d1818eb7f2bd467fbf75bad5cc8e0aea => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:44.311273 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:44.409369 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0fd3e-dac1-4537-a080-cbbd4931ee40 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:44.449751 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c29319d-8d41-494e-aac0-4beee0fc4bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:44.450755 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29319d-8d41-494e-aac0-4beee0fc4bce None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:44.451267 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29319d-8d41-494e-aac0-4beee0fc4bce None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:44.451738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29319d-8d41-494e-aac0-4beee0fc4bce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:44.534437 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c29319d-8d41-494e-aac0-4beee0fc4bce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:44.536341 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c29319d-8d41-494e-aac0-4beee0fc4bce None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:44.557265 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:44.641960 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 108/206] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:53:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:53:44.647781 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b4228-f486-4830-9c91-97ad6a857a47 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:44.793986 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b4228-f486-4830-9c91-97ad6a857a47 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:44.832731 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2466d04f-1b55-4b80-aed7-cb84790fcbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:44.833748 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466d04f-1b55-4b80-aed7-cb84790fcbb8 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:44.834228 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466d04f-1b55-4b80-aed7-cb84790fcbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:44.834715 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466d04f-1b55-4b80-aed7-cb84790fcbb8 None admin] PATH_INFO: `/v3/projects/3b402af2708b41df849963f81e65797e` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:44.836233 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2466d04f-1b55-4b80-aed7-cb84790fcbb8 None admin] RBAC: Authorizing `identity:get_project(project_id=3b402af2708b41df849963f81e65797e)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:44.837905 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2466d04f-1b55-4b80-aed7-cb84790fcbb8 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:44.840109 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 99/207] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:53:44 2022] GET /identity/v3/projects/3b402af2708b41df849963f81e65797e => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:44.852195 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c29319d-8d41-494e-aac0-4beee0fc4bce None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:44.965203 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c29319d-8d41-494e-aac0-4beee0fc4bce None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:44.983162 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:45.010238 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c723cc-413d-43d9-b141-191db666ee4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:45.010866 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c723cc-413d-43d9-b141-191db666ee4e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:45.011150 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c723cc-413d-43d9-b141-191db666ee4e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:45.011418 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c723cc-413d-43d9-b141-191db666ee4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:45.012135 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c723cc-413d-43d9-b141-191db666ee4e None admin] RBAC: Adding query filter params (role.id=06e98804662441c88b8871c32b9468aa, scope.project.id=3b402af2708b41df849963f81e65797e, user.id=ed513a0aabf84899951cac87303e0a8f) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:45.012435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c723cc-413d-43d9-b141-191db666ee4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:45.013933 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c723cc-413d-43d9-b141-191db666ee4e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:45.020017 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 109/208] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:53:44 2022] GET /identity/v3/role_assignments?user.id=ed513a0aabf84899951cac87303e0a8f&scope.project.id=3b402af2708b41df849963f81e65797e&role.id=06e98804662441c88b8871c32b9468aa => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:45.791850 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a08e7f-bb1d-455b-9c1c-f77afe1baddd None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:45.792398 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a08e7f-bb1d-455b-9c1c-f77afe1baddd None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:45.792398 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a08e7f-bb1d-455b-9c1c-f77afe1baddd None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:45.794048 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 100/209] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:45.800308 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09e9d2-720f-4263-ae19-d187e3f868d9 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:45.800745 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09e9d2-720f-4263-ae19-d187e3f868d9 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:45.800854 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09e9d2-720f-4263-ae19-d187e3f868d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:45.819885 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-7a09e9d2-720f-4263-ae19-d187e3f868d9 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:45.821113 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7a09e9d2-720f-4263-ae19-d187e3f868d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:45.847066 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9687087-c164-44f9-b7db-e213b841ded2 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:45.847351 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9687087-c164-44f9-b7db-e213b841ded2 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:45.847631 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9687087-c164-44f9-b7db-e213b841ded2 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:45.848980 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 101/210] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:45.854021 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113fa0a6-b510-4f39-9766-2babdbd32da7 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:45.854531 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113fa0a6-b510-4f39-9766-2babdbd32da7 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:45.855047 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113fa0a6-b510-4f39-9766-2babdbd32da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:45.874569 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-113fa0a6-b510-4f39-9766-2babdbd32da7 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:45.877513 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-113fa0a6-b510-4f39-9766-2babdbd32da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:45.893518 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 110/211] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:45.921744 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:45.952005 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 102/212] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:46.179298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a09e9d2-720f-4263-ae19-d187e3f868d9 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:46.222926 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113fa0a6-b510-4f39-9766-2babdbd32da7 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:46.298973 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a09e9d2-720f-4263-ae19-d187e3f868d9 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:46.318936 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:46.348830 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113fa0a6-b510-4f39-9766-2babdbd32da7 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:46.350896 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe70bbec-7904-455e-b41a-23202155e41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:46.352693 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70bbec-7904-455e-b41a-23202155e41c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:46.353181 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70bbec-7904-455e-b41a-23202155e41c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:46.353649 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70bbec-7904-455e-b41a-23202155e41c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:46.355027 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe70bbec-7904-455e-b41a-23202155e41c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:46.356745 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe70bbec-7904-455e-b41a-23202155e41c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:46.359337 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 111/213] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:53:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:46.364954 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe70bbec-7904-455e-b41a-23202155e41c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:46.392727 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ace1a6-82ae-4926-831c-75d6814e4766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:46.393363 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ace1a6-82ae-4926-831c-75d6814e4766 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:46.393639 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ace1a6-82ae-4926-831c-75d6814e4766 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:46.394029 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ace1a6-82ae-4926-831c-75d6814e4766 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:46.404653 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ace1a6-82ae-4926-831c-75d6814e4766 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:53:46.410473 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-94ace1a6-82ae-4926-831c-75d6814e4766 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:53:46.410473 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:53:46.410473 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:53:46.410473 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:53:46.410473 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:53:46.410473 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:53:46.410473 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:53:46.410473 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:53:46.410473 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:53:46.410473 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:53:46.410473 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:53:46.410473 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:53:46.410473 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:53:46.410473 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:53:46.410473 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:53:46.410473 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:53:46.411417 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:53:46.411417 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:53:46.411417 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:53:46.411417 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:53:46.411417 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:53:46.411417 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:53:46.411417 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:53:46.411417 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:53:46.411417 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:53:46.411417 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:53:46.411417 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:53:46.411417 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:53:46.411417 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:53:46.411417 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:53:46.411417 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:53:46.411417 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:53:46.412204 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:53:46.412204 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:53:46.412204 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:53:46.412204 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:53:46.412204 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:53:46.412204 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:53:46.412204 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:53:46.412204 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:53:46.412204 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:53:46.412204 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:53:46.412204 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:53:46.412204 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:53:46.416846 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 103/214] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:53:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:53:46.417742 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ace1a6-82ae-4926-831c-75d6814e4766 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:46.451660 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:46.510450 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe70bbec-7904-455e-b41a-23202155e41c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:46.528357 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ace1a6-82ae-4926-831c-75d6814e4766 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:46.560484 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307829a-6572-485d-af76-ca9f3103e9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:46.561540 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307829a-6572-485d-af76-ca9f3103e9d4 None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:46.562065 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307829a-6572-485d-af76-ca9f3103e9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:46.562573 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307829a-6572-485d-af76-ca9f3103e9d4 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:46.564296 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2307829a-6572-485d-af76-ca9f3103e9d4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:46.564952 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded504d-95b6-4303-a671-8014c2ccd5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:46.565661 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded504d-95b6-4303-a671-8014c2ccd5bb None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:46.566036 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded504d-95b6-4303-a671-8014c2ccd5bb None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:46.566404 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded504d-95b6-4303-a671-8014c2ccd5bb None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:46.566531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2307829a-6572-485d-af76-ca9f3103e9d4 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:46.567276 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded504d-95b6-4303-a671-8014c2ccd5bb None admin] RBAC: Adding query filter params (name=member) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:46.567670 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded504d-95b6-4303-a671-8014c2ccd5bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:46.569146 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded504d-95b6-4303-a671-8014c2ccd5bb None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:46.582174 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 104/215] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 14:53:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:46.585697 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded504d-95b6-4303-a671-8014c2ccd5bb None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:46.598987 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 112/216] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 14:53:46 2022] POST /identity/v3/users => generated 297 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:53:46.715964 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:46.724997 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded504d-95b6-4303-a671-8014c2ccd5bb None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:46.765508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d48df9-193c-47f2-8435-8a82e04391f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:46.767123 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d48df9-193c-47f2-8435-8a82e04391f9 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:46.767606 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d48df9-193c-47f2-8435-8a82e04391f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:46.768078 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d48df9-193c-47f2-8435-8a82e04391f9 None admin] PATH_INFO: `/v3/users/6a624b32bd1945c883b0b5be93482487` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:46.784582 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d48df9-193c-47f2-8435-8a82e04391f9 None admin] RBAC: Authorizing `identity:get_user(user_id=6a624b32bd1945c883b0b5be93482487)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:46.786094 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d48df9-193c-47f2-8435-8a82e04391f9 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:46.788214 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 105/217] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:53:46 2022] GET /identity/v3/users/6a624b32bd1945c883b0b5be93482487 => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:53:46.794213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307829a-6572-485d-af76-ca9f3103e9d4 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:46.802257 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af3652-3c6f-46df-bf7a-cb194f7d1a6d None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:46.802923 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af3652-3c6f-46df-bf7a-cb194f7d1a6d None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:46.802923 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af3652-3c6f-46df-bf7a-cb194f7d1a6d None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:46.804412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 106/218] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:53:46.810297 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2246fa-4949-4847-9c44-8affd36255fe None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:46.810598 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2246fa-4949-4847-9c44-8affd36255fe None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:46.810902 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2246fa-4949-4847-9c44-8affd36255fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:46.837958 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-fb2246fa-4949-4847-9c44-8affd36255fe None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:46.839226 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-fb2246fa-4949-4847-9c44-8affd36255fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:46.844237 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:46.927246 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 107/219] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:53:46.928958 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0d26a-758a-48a1-9392-daec348027a2 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:46.929245 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0d26a-758a-48a1-9392-daec348027a2 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:46.929524 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0d26a-758a-48a1-9392-daec348027a2 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:46.930323 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307829a-6572-485d-af76-ca9f3103e9d4 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:46.930837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 108/220] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:46.936512 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647414d0-4111-46ec-b25f-a3fc6926514e None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:46.936801 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647414d0-4111-46ec-b25f-a3fc6926514e None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:46.937093 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647414d0-4111-46ec-b25f-a3fc6926514e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:46.960214 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-647414d0-4111-46ec-b25f-a3fc6926514e None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:46.960430 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-647414d0-4111-46ec-b25f-a3fc6926514e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:46.973356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e8c286-a371-4fdb-9296-58ae0f5e2c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:46.974284 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8c286-a371-4fdb-9296-58ae0f5e2c0a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:46.974742 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8c286-a371-4fdb-9296-58ae0f5e2c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:46.975226 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8c286-a371-4fdb-9296-58ae0f5e2c0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:47.039987 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 109/221] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:53:47.066374 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e8c286-a371-4fdb-9296-58ae0f5e2c0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:47.067935 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e8c286-a371-4fdb-9296-58ae0f5e2c0a None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:47.103482 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:47.179112 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 113/222] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:53:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:53:47.180641 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647414d0-4111-46ec-b25f-a3fc6926514e None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:47.203227 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:47.243517 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e8c286-a371-4fdb-9296-58ae0f5e2c0a None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:47.292195 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647414d0-4111-46ec-b25f-a3fc6926514e None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:47.330563 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2fe2f-f090-477b-82fc-2fd88a3361c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:47.331487 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2fe2f-f090-477b-82fc-2fd88a3361c3 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:47.331858 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2fe2f-f090-477b-82fc-2fd88a3361c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:47.332225 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2fe2f-f090-477b-82fc-2fd88a3361c3 None admin] PATH_INFO: `/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:47.333454 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba2fe2f-f090-477b-82fc-2fd88a3361c3 None admin] RBAC: Authorizing `identity:get_project(project_id=8f8cdaafdd52411bb20efe51ddc2565c)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:47.335083 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba2fe2f-f090-477b-82fc-2fd88a3361c3 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:47.337126 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 110/223] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:53:47 2022] GET /identity/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:47.343761 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2fe2f-f090-477b-82fc-2fd88a3361c3 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:47.377081 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e8c286-a371-4fdb-9296-58ae0f5e2c0a None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:47.421415 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9132349-248b-4609-9600-9ee3fd5747fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:47.422365 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9132349-248b-4609-9600-9ee3fd5747fe None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:47.422652 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9132349-248b-4609-9600-9ee3fd5747fe None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:47.423040 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9132349-248b-4609-9600-9ee3fd5747fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:47.427484 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:47.437693 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9132349-248b-4609-9600-9ee3fd5747fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:53:47.443423 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-d9132349-248b-4609-9600-9ee3fd5747fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:53:47.443423 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:53:47.443423 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:53:47.443423 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:53:47.443423 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:53:47.443423 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:53:47.443423 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:53:47.443423 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:53:47.443423 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:53:47.443423 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:53:47.443423 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:53:47.443423 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:53:47.443423 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:53:47.443423 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:53:47.443423 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:53:47.443423 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:53:47.444525 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:53:47.444525 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:53:47.444525 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:53:47.444525 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:53:47.444525 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:53:47.444525 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:53:47.444525 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:53:47.444525 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:53:47.444525 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:53:47.444525 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:53:47.444525 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:53:47.444525 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:53:47.444525 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:53:47.444525 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:53:47.444525 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:53:47.444525 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:53:47.445431 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:53:47.445431 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:53:47.445431 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:53:47.445431 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:53:47.445431 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:53:47.445431 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:53:47.445431 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:53:47.445431 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:53:47.445431 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:53:47.445431 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:53:47.445431 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:53:47.445431 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:53:47.445431 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 114/224] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:53:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:53:47.445431 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9132349-248b-4609-9600-9ee3fd5747fe None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:47.456779 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:47.487836 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2fe2f-f090-477b-82fc-2fd88a3361c3 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:47.531106 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa8dcdf-0db5-440a-bd39-23a611f69f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:47.532564 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8dcdf-0db5-440a-bd39-23a611f69f31 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:47.533012 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8dcdf-0db5-440a-bd39-23a611f69f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:47.533432 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8dcdf-0db5-440a-bd39-23a611f69f31 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:47.548941 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa8dcdf-0db5-440a-bd39-23a611f69f31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:53:47.555406 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-aaa8dcdf-0db5-440a-bd39-23a611f69f31 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:53:47.555406 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:53:47.555406 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:53:47.555406 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:53:47.555406 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:53:47.555406 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:53:47.555406 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:53:47.555406 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:53:47.555406 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:53:47.555406 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:53:47.555406 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:53:47.555406 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:53:47.555406 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:53:47.555406 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:53:47.555406 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:53:47.555406 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:53:47.556482 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:53:47.556482 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:53:47.556482 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:53:47.556482 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:53:47.556482 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:53:47.556482 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:53:47.556482 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:53:47.556482 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:53:47.556482 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:53:47.556482 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:53:47.556482 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:53:47.556482 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:53:47.556482 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:53:47.556482 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:53:47.556482 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:53:47.556482 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:53:47.557461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:53:47.557461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:53:47.557461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:53:47.557461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:53:47.557461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:53:47.557461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:53:47.557461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:53:47.557461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:53:47.557461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:53:47.557461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:53:47.557461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:53:47.557461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:53:47.558336 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9132349-248b-4609-9600-9ee3fd5747fe None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:47.559821 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 111/225] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 14:53:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:53:47.561778 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa8dcdf-0db5-440a-bd39-23a611f69f31 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:47.595956 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47119caf-4ac6-4485-810e-8a28c437333b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:47.596661 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47119caf-4ac6-4485-810e-8a28c437333b None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:47.596941 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47119caf-4ac6-4485-810e-8a28c437333b None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:47.597206 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47119caf-4ac6-4485-810e-8a28c437333b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:47.597944 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47119caf-4ac6-4485-810e-8a28c437333b None admin] RBAC: Adding query filter params (role.id=c9da950c75da4939b8bd649c11015650, scope.project.id=8f8cdaafdd52411bb20efe51ddc2565c, user.id=6a624b32bd1945c883b0b5be93482487) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:47.598229 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47119caf-4ac6-4485-810e-8a28c437333b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:47.599719 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47119caf-4ac6-4485-810e-8a28c437333b None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:47.605847 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 115/226] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:53:47 2022] GET /identity/v3/role_assignments?user.id=6a624b32bd1945c883b0b5be93482487&scope.project.id=8f8cdaafdd52411bb20efe51ddc2565c&role.id=c9da950c75da4939b8bd649c11015650 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:47.606654 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47119caf-4ac6-4485-810e-8a28c437333b None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:47.694694 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa8dcdf-0db5-440a-bd39-23a611f69f31 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:47.741596 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b25fbb-6b5a-4462-bc97-39ed7d4bddb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:47.742618 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b25fbb-6b5a-4462-bc97-39ed7d4bddb1 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:47.743210 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b25fbb-6b5a-4462-bc97-39ed7d4bddb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:47.743685 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b25fbb-6b5a-4462-bc97-39ed7d4bddb1 None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:47.744578 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b25fbb-6b5a-4462-bc97-39ed7d4bddb1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:47.745088 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b25fbb-6b5a-4462-bc97-39ed7d4bddb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:47.746720 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b25fbb-6b5a-4462-bc97-39ed7d4bddb1 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:47.752031 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:47.758221 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 112/227] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:53:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:47.769780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b25fbb-6b5a-4462-bc97-39ed7d4bddb1 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:47.773430 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:47.841523 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47119caf-4ac6-4485-810e-8a28c437333b None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:47.886783 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c96b233-f1ee-4d9b-804e-23f9540dfe40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:47.887457 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96b233-f1ee-4d9b-804e-23f9540dfe40 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:47.887733 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96b233-f1ee-4d9b-804e-23f9540dfe40 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:47.888008 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96b233-f1ee-4d9b-804e-23f9540dfe40 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:47.888993 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c96b233-f1ee-4d9b-804e-23f9540dfe40 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:47.889328 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c96b233-f1ee-4d9b-804e-23f9540dfe40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:47.891987 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c96b233-f1ee-4d9b-804e-23f9540dfe40 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:47.893744 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b25fbb-6b5a-4462-bc97-39ed7d4bddb1 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:47.902195 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 116/228] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:53:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:47.912922 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c96b233-f1ee-4d9b-804e-23f9540dfe40 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:47.958053 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10873fa5-ddf5-4275-9441-e9c6e59239ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:47.959159 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10873fa5-ddf5-4275-9441-e9c6e59239ee None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:47.960578 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10873fa5-ddf5-4275-9441-e9c6e59239ee None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:47.961075 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10873fa5-ddf5-4275-9441-e9c6e59239ee None admin] PATH_INFO: `/v3/users/91b926a36a5f4024a2cd0788d2867305` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:47.962512 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10873fa5-ddf5-4275-9441-e9c6e59239ee None admin] RBAC: Authorizing `identity:get_user(user_id=91b926a36a5f4024a2cd0788d2867305)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:47.965642 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10873fa5-ddf5-4275-9441-e9c6e59239ee None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:47.968297 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 113/229] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:53:47 2022] GET /identity/v3/users/91b926a36a5f4024a2cd0788d2867305 => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:53:47.974268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10873fa5-ddf5-4275-9441-e9c6e59239ee None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:48.068747 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c96b233-f1ee-4d9b-804e-23f9540dfe40 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:48.095292 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:48.109805 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3922c6dd-b461-4f72-bbd6-f4ae38a416ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:48.110489 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3922c6dd-b461-4f72-bbd6-f4ae38a416ce None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:48.110771 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3922c6dd-b461-4f72-bbd6-f4ae38a416ce None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:48.111156 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3922c6dd-b461-4f72-bbd6-f4ae38a416ce None admin] PATH_INFO: `/v3/users/91b926a36a5f4024a2cd0788d2867305` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:48.112365 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3922c6dd-b461-4f72-bbd6-f4ae38a416ce None admin] RBAC: Authorizing `identity:get_user(user_id=91b926a36a5f4024a2cd0788d2867305)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:48.113901 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3922c6dd-b461-4f72-bbd6-f4ae38a416ce None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:48.115994 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 117/230] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:53:47 2022] GET /identity/v3/users/91b926a36a5f4024a2cd0788d2867305 => generated 283 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:53:48.121345 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3922c6dd-b461-4f72-bbd6-f4ae38a416ce None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:48.182366 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:48.238668 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3922c6dd-b461-4f72-bbd6-f4ae38a416ce None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:48.266754 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10873fa5-ddf5-4275-9441-e9c6e59239ee None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:48.283369 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ecb37-9354-4959-b876-5b23128e8f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:48.285151 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ecb37-9354-4959-b876-5b23128e8f9d None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:48.286285 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ecb37-9354-4959-b876-5b23128e8f9d None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:48.286753 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ecb37-9354-4959-b876-5b23128e8f9d None admin] PATH_INFO: `/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:48.289967 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ecb37-9354-4959-b876-5b23128e8f9d None admin] RBAC: Authorizing `identity:get_project(project_id=8f8cdaafdd52411bb20efe51ddc2565c)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:48.293324 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ecb37-9354-4959-b876-5b23128e8f9d None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:48.296224 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 118/231] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:53:48 2022] GET /identity/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:53:48.303289 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ecb37-9354-4959-b876-5b23128e8f9d None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:48.338521 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34eea63-8ed5-4564-81f7-986c19c56847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:48.340139 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34eea63-8ed5-4564-81f7-986c19c56847 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:48.340912 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34eea63-8ed5-4564-81f7-986c19c56847 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:48.341722 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34eea63-8ed5-4564-81f7-986c19c56847 None admin] PATH_INFO: `/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:48.345896 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34eea63-8ed5-4564-81f7-986c19c56847 None admin] RBAC: Authorizing `identity:get_project(project_id=8f8cdaafdd52411bb20efe51ddc2565c)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:48.350074 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34eea63-8ed5-4564-81f7-986c19c56847 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:48.353154 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 114/232] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:53:47 2022] GET /identity/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c => generated 302 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:48.358633 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34eea63-8ed5-4564-81f7-986c19c56847 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:48.451161 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:48.515229 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ecb37-9354-4959-b876-5b23128e8f9d None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:48.555621 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb436ced-8ead-4ca9-b2de-abc331de4bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:48.556429 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb436ced-8ead-4ca9-b2de-abc331de4bd6 None admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:48.556794 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb436ced-8ead-4ca9-b2de-abc331de4bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:48.557126 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb436ced-8ead-4ca9-b2de-abc331de4bd6 None admin] PATH_INFO: `/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c/users/91b926a36a5f4024a2cd0788d2867305/roles/d1818eb7f2bd467fbf75bad5cc8e0aea` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:48.560427 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb436ced-8ead-4ca9-b2de-abc331de4bd6 None admin] RBAC: Authorizing `identity:create_grant(project_id=8f8cdaafdd52411bb20efe51ddc2565c, user_id=91b926a36a5f4024a2cd0788d2867305, role_id=d1818eb7f2bd467fbf75bad5cc8e0aea)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:48.561889 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb436ced-8ead-4ca9-b2de-abc331de4bd6 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:48.565054 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34eea63-8ed5-4564-81f7-986c19c56847 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:48.581545 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 119/233] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 14:53:48 2022] PUT /identity/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c/users/91b926a36a5f4024a2cd0788d2867305/roles/d1818eb7f2bd467fbf75bad5cc8e0aea => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 14:53:48.643714 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33032105-ac18-41e7-9b8a-4607f870e492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:48.644338 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33032105-ac18-41e7-9b8a-4607f870e492 None admin] REQUEST_METHOD: `PUT` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:48.644628 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33032105-ac18-41e7-9b8a-4607f870e492 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:48.644905 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33032105-ac18-41e7-9b8a-4607f870e492 None admin] PATH_INFO: `/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c/users/91b926a36a5f4024a2cd0788d2867305/roles/99e8fe7df8e7446680a81d4eaaa1cb68` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:48.647145 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33032105-ac18-41e7-9b8a-4607f870e492 None admin] RBAC: Authorizing `identity:create_grant(project_id=8f8cdaafdd52411bb20efe51ddc2565c, user_id=91b926a36a5f4024a2cd0788d2867305, role_id=99e8fe7df8e7446680a81d4eaaa1cb68)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:48.648652 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33032105-ac18-41e7-9b8a-4607f870e492 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:48.666162 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:48.672003 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 115/234] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 14:53:48 2022] PUT /identity/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c/users/91b926a36a5f4024a2cd0788d2867305/roles/99e8fe7df8e7446680a81d4eaaa1cb68 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:53:49.159310 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74693928-573f-413d-8358-281468b23c7a None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:49.159821 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74693928-573f-413d-8358-281468b23c7a None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:49.159821 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74693928-573f-413d-8358-281468b23c7a None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:49.161346 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 120/235] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:49.166684 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d1579-186a-4a47-a90e-e22cee4518bd None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:49.167162 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d1579-186a-4a47-a90e-e22cee4518bd None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:49.167337 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d1579-186a-4a47-a90e-e22cee4518bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:49.187311 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-718d1579-186a-4a47-a90e-e22cee4518bd None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:49.188653 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-718d1579-186a-4a47-a90e-e22cee4518bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:49.279062 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 116/236] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:49.658985 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acc3d8-ee4b-4cd7-aaf1-cdde279424e3 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:49.659326 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acc3d8-ee4b-4cd7-aaf1-cdde279424e3 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:49.659500 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acc3d8-ee4b-4cd7-aaf1-cdde279424e3 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:49.661103 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 121/237] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:49.666348 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771187ba-e1a8-48d7-b094-e010ae224104 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:49.666938 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771187ba-e1a8-48d7-b094-e010ae224104 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:49.667415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771187ba-e1a8-48d7-b094-e010ae224104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:49.691506 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-771187ba-e1a8-48d7-b094-e010ae224104 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:49.691806 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15acc3d8-ee4b-4cd7-aaf1-cdde279424e3 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:49.693239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-771187ba-e1a8-48d7-b094-e010ae224104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:49.819431 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 117/238] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:49.859613 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15acc3d8-ee4b-4cd7-aaf1-cdde279424e3 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:49.883237 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:49.906980 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d17cf-03ae-479f-8cfd-1d7df1d07977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:49.907835 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d17cf-03ae-479f-8cfd-1d7df1d07977 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:49.908121 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d17cf-03ae-479f-8cfd-1d7df1d07977 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:49.908397 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d17cf-03ae-479f-8cfd-1d7df1d07977 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:49.925943 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63d17cf-03ae-479f-8cfd-1d7df1d07977 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:53:49.930751 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-c63d17cf-03ae-479f-8cfd-1d7df1d07977 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:53:49.930751 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:53:49.930751 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:53:49.930751 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:53:49.930751 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:53:49.930751 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:53:49.930751 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:53:49.930751 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:53:49.930751 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:53:49.930751 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:53:49.930751 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:53:49.930751 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:53:49.930751 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:53:49.930751 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:53:49.930751 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:53:49.930751 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:53:49.931645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:53:49.931645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:53:49.931645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:53:49.931645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:53:49.931645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:53:49.931645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:53:49.931645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:53:49.931645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:53:49.931645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:53:49.931645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:53:49.931645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:53:49.931645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:53:49.931645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:53:49.931645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:53:49.931645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:53:49.931645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:53:49.933091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:53:49.933091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:53:49.933091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:53:49.933091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:53:49.933091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:53:49.933091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:53:49.933091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:53:49.933091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:53:49.933091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:53:49.933091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:53:49.933091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:53:49.933091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:53:49.933091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 122/239] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:53:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:53:49.938243 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771187ba-e1a8-48d7-b094-e010ae224104 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:50.036463 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:50.081636 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771187ba-e1a8-48d7-b094-e010ae224104 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:50.130041 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d17cf-03ae-479f-8cfd-1d7df1d07977 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:50.132334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97665e-4db8-4620-941b-9ea6208424b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:50.132970 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97665e-4db8-4620-941b-9ea6208424b9 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:50.133250 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97665e-4db8-4620-941b-9ea6208424b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:50.133523 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97665e-4db8-4620-941b-9ea6208424b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:50.134208 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97665e-4db8-4620-941b-9ea6208424b9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:50.134509 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97665e-4db8-4620-941b-9ea6208424b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:50.135960 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97665e-4db8-4620-941b-9ea6208424b9 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:50.145810 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 118/240] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 14:53:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:50.151346 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97665e-4db8-4620-941b-9ea6208424b9 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:50.255790 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d17cf-03ae-479f-8cfd-1d7df1d07977 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:50.279870 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97665e-4db8-4620-941b-9ea6208424b9 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:50.300593 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e828eb-0ed1-4163-8252-d68409c07425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:50.301228 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e828eb-0ed1-4163-8252-d68409c07425 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:50.301569 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e828eb-0ed1-4163-8252-d68409c07425 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:50.301848 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e828eb-0ed1-4163-8252-d68409c07425 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:50.302984 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e828eb-0ed1-4163-8252-d68409c07425 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:50.304455 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e828eb-0ed1-4163-8252-d68409c07425 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:50.306217 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:50.306389 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 123/241] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:53:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:50.307835 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac5473-21a0-4d6b-88fa-321eeda5c27f None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:50.308120 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac5473-21a0-4d6b-88fa-321eeda5c27f None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:50.309030 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac5473-21a0-4d6b-88fa-321eeda5c27f None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:50.309593 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 124/242] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:50.314644 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac5473-21a0-4d6b-88fa-321eeda5c27f None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:50.330240 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec22887-f5f2-44bb-ba70-e8b0a011b26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:50.331354 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec22887-f5f2-44bb-ba70-e8b0a011b26d None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:50.331907 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec22887-f5f2-44bb-ba70-e8b0a011b26d None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:50.332435 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec22887-f5f2-44bb-ba70-e8b0a011b26d None admin] PATH_INFO: `/v3/users/aa385023402746d5ad582a05e3039347` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:50.351010 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec22887-f5f2-44bb-ba70-e8b0a011b26d None admin] RBAC: Authorizing `identity:get_user(user_id=aa385023402746d5ad582a05e3039347)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:50.353205 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec22887-f5f2-44bb-ba70-e8b0a011b26d None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:50.356581 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 119/243] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:53:50 2022] GET /identity/v3/users/aa385023402746d5ad582a05e3039347 => generated 297 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:53:50.358047 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae0123-edc7-49ee-92d9-ba0f27f404f7 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:50.359090 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae0123-edc7-49ee-92d9-ba0f27f404f7 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:50.359730 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae0123-edc7-49ee-92d9-ba0f27f404f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:50.370671 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:50.385444 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-1bae0123-edc7-49ee-92d9-ba0f27f404f7 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:50.387224 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-1bae0123-edc7-49ee-92d9-ba0f27f404f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:50.432519 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac5473-21a0-4d6b-88fa-321eeda5c27f None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:50.466087 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 120/244] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:50.466999 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bae0123-edc7-49ee-92d9-ba0f27f404f7 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:50.493063 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7f4180-2c59-4b55-a48f-cb7c71eb1ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:50.493682 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f4180-2c59-4b55-a48f-cb7c71eb1ff3 None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:50.493960 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f4180-2c59-4b55-a48f-cb7c71eb1ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:50.494232 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f4180-2c59-4b55-a48f-cb7c71eb1ff3 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:50.494924 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7f4180-2c59-4b55-a48f-cb7c71eb1ff3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:50.497624 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7f4180-2c59-4b55-a48f-cb7c71eb1ff3 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:50.536853 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 125/245] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 14:53:50 2022] POST /identity/v3/users => generated 293 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:53:50.605307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bae0123-edc7-49ee-92d9-ba0f27f404f7 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:50.615721 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:50.642884 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42416118-b66f-4922-9c65-e336c26cac70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:50.643516 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42416118-b66f-4922-9c65-e336c26cac70 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:50.643796 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42416118-b66f-4922-9c65-e336c26cac70 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:50.644068 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42416118-b66f-4922-9c65-e336c26cac70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:50.735480 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42416118-b66f-4922-9c65-e336c26cac70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:50.737154 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42416118-b66f-4922-9c65-e336c26cac70 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:50.867308 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 121/246] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:53:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:53:50.872857 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7f4180-2c59-4b55-a48f-cb7c71eb1ff3 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:50.918386 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42416118-b66f-4922-9c65-e336c26cac70 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:50.930030 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:51.005282 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7f4180-2c59-4b55-a48f-cb7c71eb1ff3 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:51.007027 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:51.059560 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57e5834-cb06-4533-affb-8cdacec30921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:51.060626 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e5834-cb06-4533-affb-8cdacec30921 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:51.061165 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e5834-cb06-4533-affb-8cdacec30921 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:51.061636 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e5834-cb06-4533-affb-8cdacec30921 None admin] PATH_INFO: `/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:51.063656 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57e5834-cb06-4533-affb-8cdacec30921 None admin] RBAC: Authorizing `identity:get_project(project_id=8f8cdaafdd52411bb20efe51ddc2565c)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:51.067313 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57e5834-cb06-4533-affb-8cdacec30921 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:51.068060 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42416118-b66f-4922-9c65-e336c26cac70 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:51.069630 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 126/247] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:53:50 2022] GET /identity/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:51.075330 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57e5834-cb06-4533-affb-8cdacec30921 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:51.109681 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab108e-fdee-4212-9f25-a2aed6b4726c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:51.110399 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab108e-fdee-4212-9f25-a2aed6b4726c None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:51.110837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab108e-fdee-4212-9f25-a2aed6b4726c None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:51.111251 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab108e-fdee-4212-9f25-a2aed6b4726c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:51.123204 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eab108e-fdee-4212-9f25-a2aed6b4726c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:53:51.130154 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-3eab108e-fdee-4212-9f25-a2aed6b4726c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:53:51.130154 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:53:51.130154 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:53:51.130154 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:53:51.130154 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:53:51.130154 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:53:51.130154 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:53:51.130154 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:53:51.130154 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:53:51.130154 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:53:51.130154 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:53:51.130154 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:53:51.130154 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:53:51.130154 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:53:51.130154 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:53:51.130154 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:53:51.131101 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:53:51.131101 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:53:51.131101 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:53:51.131101 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:53:51.131101 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:53:51.131101 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:53:51.131101 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:53:51.131101 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:53:51.131101 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:53:51.131101 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:53:51.131101 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:53:51.131101 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:53:51.131101 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:53:51.131101 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:53:51.131101 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:53:51.131101 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:53:51.132329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:53:51.132329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:53:51.132329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:53:51.132329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:53:51.132329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:53:51.132329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:53:51.132329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:53:51.132329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:53:51.132329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:53:51.132329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:53:51.132329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:53:51.132329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:53:51.132329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 122/248] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:53:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:53:51.137248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab108e-fdee-4212-9f25-a2aed6b4726c None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:51.197861 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57e5834-cb06-4533-affb-8cdacec30921 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:51.212724 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:51.241289 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95aaf9e-bf50-4818-a5f1-01f5d8753f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:51.242358 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95aaf9e-bf50-4818-a5f1-01f5d8753f00 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:51.242948 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95aaf9e-bf50-4818-a5f1-01f5d8753f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:51.243762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95aaf9e-bf50-4818-a5f1-01f5d8753f00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:51.244867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95aaf9e-bf50-4818-a5f1-01f5d8753f00 None admin] RBAC: Adding query filter params (role.id=06e98804662441c88b8871c32b9468aa, scope.project.id=8f8cdaafdd52411bb20efe51ddc2565c, user.id=aa385023402746d5ad582a05e3039347) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:51.245417 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95aaf9e-bf50-4818-a5f1-01f5d8753f00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:51.247280 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95aaf9e-bf50-4818-a5f1-01f5d8753f00 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:51.255436 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 127/249] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:53:51 2022] GET /identity/v3/role_assignments?user.id=aa385023402746d5ad582a05e3039347&scope.project.id=8f8cdaafdd52411bb20efe51ddc2565c&role.id=06e98804662441c88b8871c32b9468aa => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:51.256750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab108e-fdee-4212-9f25-a2aed6b4726c None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:51.306525 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5b4ea-114b-4ff5-9a33-18b7f627c61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:51.307193 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5b4ea-114b-4ff5-9a33-18b7f627c61d None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:51.307458 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5b4ea-114b-4ff5-9a33-18b7f627c61d None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:51.307984 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5b4ea-114b-4ff5-9a33-18b7f627c61d None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:51.308669 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a5b4ea-114b-4ff5-9a33-18b7f627c61d None admin] RBAC: Adding query filter params (name=member) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:51.308973 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a5b4ea-114b-4ff5-9a33-18b7f627c61d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:51.310388 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a5b4ea-114b-4ff5-9a33-18b7f627c61d None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:51.321047 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 123/250] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 14:53:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:53:51.326824 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95aaf9e-bf50-4818-a5f1-01f5d8753f00 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:51.368381 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:51.408203 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2855351-92cb-4722-8eef-819774cdafd5 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:51.408385 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2855351-92cb-4722-8eef-819774cdafd5 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:51.408662 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2855351-92cb-4722-8eef-819774cdafd5 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:51.410345 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 124/251] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:53:51.417044 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853f9b7-672d-4d57-86e8-36fce3b05466 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:51.417380 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853f9b7-672d-4d57-86e8-36fce3b05466 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:51.417601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853f9b7-672d-4d57-86e8-36fce3b05466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:51.444482 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-5853f9b7-672d-4d57-86e8-36fce3b05466 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:51.445722 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-5853f9b7-672d-4d57-86e8-36fce3b05466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:51.463421 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95aaf9e-bf50-4818-a5f1-01f5d8753f00 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:51.504557 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e3e40-cf31-4da5-b23d-af3c6e03f232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:51.505605 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e3e40-cf31-4da5-b23d-af3c6e03f232 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:51.506251 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e3e40-cf31-4da5-b23d-af3c6e03f232 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:51.506746 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e3e40-cf31-4da5-b23d-af3c6e03f232 None admin] PATH_INFO: `/v3/users/6a624b32bd1945c883b0b5be93482487` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:51.508249 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050e3e40-cf31-4da5-b23d-af3c6e03f232 None admin] RBAC: Authorizing `identity:get_user(user_id=6a624b32bd1945c883b0b5be93482487)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:51.509923 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050e3e40-cf31-4da5-b23d-af3c6e03f232 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:51.512122 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 128/252] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:53:51 2022] GET /identity/v3/users/6a624b32bd1945c883b0b5be93482487 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:53:51.518235 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e3e40-cf31-4da5-b23d-af3c6e03f232 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:51.539627 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 125/253] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:51.555369 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae247cc2-8459-44df-95cb-0877f2772f68 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:51.555927 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae247cc2-8459-44df-95cb-0877f2772f68 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:51.556379 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae247cc2-8459-44df-95cb-0877f2772f68 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:51.558441 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 126/254] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:53:51.564717 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca51d5-087b-4dfe-808f-430f3c4987a5 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:51.565227 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca51d5-087b-4dfe-808f-430f3c4987a5 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:51.565676 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca51d5-087b-4dfe-808f-430f3c4987a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:51.597814 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-ccca51d5-087b-4dfe-808f-430f3c4987a5 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:51.599704 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-ccca51d5-087b-4dfe-808f-430f3c4987a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:51.644870 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e3e40-cf31-4da5-b23d-af3c6e03f232 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:51.647021 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:51.708404 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:51.716521 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663082f6-25fb-4e4d-89fc-bc899af00cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:51.717572 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663082f6-25fb-4e4d-89fc-bc899af00cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:51.717856 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663082f6-25fb-4e4d-89fc-bc899af00cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:51.718134 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663082f6-25fb-4e4d-89fc-bc899af00cc3 None admin] PATH_INFO: `/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:51.719986 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663082f6-25fb-4e4d-89fc-bc899af00cc3 None admin] RBAC: Authorizing `identity:get_project(project_id=8f8cdaafdd52411bb20efe51ddc2565c)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:51.721719 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663082f6-25fb-4e4d-89fc-bc899af00cc3 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:51.723421 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 129/255] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:53:51 2022] GET /identity/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:53:51.731258 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663082f6-25fb-4e4d-89fc-bc899af00cc3 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:51.734252 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 127/256] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:53:51.868951 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663082f6-25fb-4e4d-89fc-bc899af00cc3 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:51.915582 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8739ec-5df0-4086-84e5-222d042ef117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:51.916688 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8739ec-5df0-4086-84e5-222d042ef117 None admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:51.917188 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8739ec-5df0-4086-84e5-222d042ef117 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:51.917657 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8739ec-5df0-4086-84e5-222d042ef117 None admin] PATH_INFO: `/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c/users/6a624b32bd1945c883b0b5be93482487/roles/c9da950c75da4939b8bd649c11015650` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:51.919856 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8739ec-5df0-4086-84e5-222d042ef117 None admin] RBAC: Authorizing `identity:create_grant(project_id=8f8cdaafdd52411bb20efe51ddc2565c, user_id=6a624b32bd1945c883b0b5be93482487, role_id=c9da950c75da4939b8bd649c11015650)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:51.921587 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8739ec-5df0-4086-84e5-222d042ef117 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:51.944611 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 130/257] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 14:53:51 2022] PUT /identity/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c/users/6a624b32bd1945c883b0b5be93482487/roles/c9da950c75da4939b8bd649c11015650 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:53:52.021569 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca51d5-087b-4dfe-808f-430f3c4987a5 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:52.157732 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca51d5-087b-4dfe-808f-430f3c4987a5 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:52.205136 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6977cb32-e005-4ce8-9506-3afc7f31b338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:52.206142 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977cb32-e005-4ce8-9506-3afc7f31b338 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:52.206653 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977cb32-e005-4ce8-9506-3afc7f31b338 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:52.207176 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977cb32-e005-4ce8-9506-3afc7f31b338 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:52.218327 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6977cb32-e005-4ce8-9506-3afc7f31b338 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:53:52.223739 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-6977cb32-e005-4ce8-9506-3afc7f31b338 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:53:52.223739 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:53:52.223739 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:53:52.223739 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:53:52.223739 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:53:52.223739 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:53:52.223739 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:53:52.223739 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:53:52.223739 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:53:52.223739 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:53:52.223739 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:53:52.223739 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:53:52.223739 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:53:52.223739 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:53:52.223739 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:53:52.223739 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:53:52.224637 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:53:52.224637 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:53:52.224637 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:53:52.224637 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:53:52.224637 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:53:52.224637 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:53:52.224637 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:53:52.224637 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:53:52.224637 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:53:52.224637 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:53:52.224637 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:53:52.224637 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:53:52.224637 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:53:52.224637 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:53:52.224637 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:53:52.224637 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:53:52.225434 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:53:52.225434 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:53:52.225434 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:53:52.225434 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:53:52.225434 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:53:52.225434 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:53:52.225434 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:53:52.225434 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:53:52.225434 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:53:52.225434 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:53:52.225434 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:53:52.225434 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:53:52.228161 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 128/258] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:53:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:53:52.233650 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8739ec-5df0-4086-84e5-222d042ef117 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:52.277241 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6977cb32-e005-4ce8-9506-3afc7f31b338 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:52.279323 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:52.372325 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:52.415512 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8739ec-5df0-4086-84e5-222d042ef117 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:52.469127 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6977cb32-e005-4ce8-9506-3afc7f31b338 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:52.484263 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd311cf-a469-43c7-b9b6-7623bd1fb00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:52.485163 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd311cf-a469-43c7-b9b6-7623bd1fb00d None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:52.485517 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd311cf-a469-43c7-b9b6-7623bd1fb00d None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:52.485792 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd311cf-a469-43c7-b9b6-7623bd1fb00d None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:52.486471 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd311cf-a469-43c7-b9b6-7623bd1fb00d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:52.486776 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd311cf-a469-43c7-b9b6-7623bd1fb00d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:52.490099 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd311cf-a469-43c7-b9b6-7623bd1fb00d None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:52.505582 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 131/259] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:53:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:52.512760 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd311cf-a469-43c7-b9b6-7623bd1fb00d None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:52.542500 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952d6c8-2c02-4d4e-b135-95b869d01070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:52.543705 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952d6c8-2c02-4d4e-b135-95b869d01070 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:52.545213 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952d6c8-2c02-4d4e-b135-95b869d01070 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:52.545546 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952d6c8-2c02-4d4e-b135-95b869d01070 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:52.572801 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2952d6c8-2c02-4d4e-b135-95b869d01070 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:53:52.579588 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-2952d6c8-2c02-4d4e-b135-95b869d01070 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:53:52.579588 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:53:52.579588 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:53:52.579588 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:53:52.579588 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:53:52.579588 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:53:52.579588 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:53:52.579588 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:53:52.579588 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:53:52.579588 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:53:52.579588 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:53:52.579588 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:53:52.579588 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:53:52.579588 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:53:52.579588 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:53:52.579588 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:53:52.580530 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:53:52.580530 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:53:52.580530 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:53:52.580530 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:53:52.580530 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:53:52.580530 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:53:52.580530 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:53:52.580530 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:53:52.580530 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:53:52.580530 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:53:52.580530 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:53:52.580530 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:53:52.580530 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:53:52.580530 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:53:52.580530 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:53:52.580530 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:53:52.581333 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:53:52.581333 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:53:52.581333 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:53:52.581333 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:53:52.581333 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:53:52.581333 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:53:52.581333 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:53:52.581333 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:53:52.581333 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:53:52.581333 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:53:52.581333 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:53:52.581333 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:53:52.586032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 129/260] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 14:53:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:53:52.588338 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952d6c8-2c02-4d4e-b135-95b869d01070 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:52.703979 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:52.718094 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd311cf-a469-43c7-b9b6-7623bd1fb00d None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:52.764760 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952d6c8-2c02-4d4e-b135-95b869d01070 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:52.788305 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:52.804466 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c670d-e75e-4877-8f67-79cebdb74cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:52.805046 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c670d-e75e-4877-8f67-79cebdb74cac None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:52.805328 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c670d-e75e-4877-8f67-79cebdb74cac None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:52.805603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c670d-e75e-4877-8f67-79cebdb74cac None admin] PATH_INFO: `/v3/users/91b926a36a5f4024a2cd0788d2867305` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:52.811744 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40c670d-e75e-4877-8f67-79cebdb74cac None admin] RBAC: Authorizing `identity:get_user(user_id=91b926a36a5f4024a2cd0788d2867305)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:52.814038 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40c670d-e75e-4877-8f67-79cebdb74cac None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:52.816329 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 132/261] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:53:52 2022] GET /identity/v3/users/91b926a36a5f4024a2cd0788d2867305 => generated 283 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:53:52.822296 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c670d-e75e-4877-8f67-79cebdb74cac None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:52.860798 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7d6af-ca14-4f17-a656-ec1372994d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:52.861428 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7d6af-ca14-4f17-a656-ec1372994d9a None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:52.861704 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7d6af-ca14-4f17-a656-ec1372994d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:52.861973 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7d6af-ca14-4f17-a656-ec1372994d9a None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:52.862646 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7d6af-ca14-4f17-a656-ec1372994d9a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:52.862971 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7d6af-ca14-4f17-a656-ec1372994d9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:52.864372 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7d6af-ca14-4f17-a656-ec1372994d9a None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:52.882079 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 130/262] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:53:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:53:52.888935 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7d6af-ca14-4f17-a656-ec1372994d9a None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:53.080869 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c670d-e75e-4877-8f67-79cebdb74cac None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:53.129591 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7d6af-ca14-4f17-a656-ec1372994d9a None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:53.164448 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde7143b-02f5-4378-b219-18e318173b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:53.165069 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7143b-02f5-4378-b219-18e318173b65 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:53.165349 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7143b-02f5-4378-b219-18e318173b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:53.165621 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7143b-02f5-4378-b219-18e318173b65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:53.183198 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfe68ea-211c-4b59-b40f-dc7e4b173c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:53.184177 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe68ea-211c-4b59-b40f-dc7e4b173c80 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:53.184680 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe68ea-211c-4b59-b40f-dc7e4b173c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:53.190878 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe68ea-211c-4b59-b40f-dc7e4b173c80 None admin] PATH_INFO: `/v3/users/91b926a36a5f4024a2cd0788d2867305` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:53.190878 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfe68ea-211c-4b59-b40f-dc7e4b173c80 None admin] RBAC: Authorizing `identity:get_user(user_id=91b926a36a5f4024a2cd0788d2867305)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:53.190878 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfe68ea-211c-4b59-b40f-dc7e4b173c80 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:53.192756 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 131/263] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:53:52 2022] GET /identity/v3/users/91b926a36a5f4024a2cd0788d2867305 => generated 283 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:53:53.198390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfe68ea-211c-4b59-b40f-dc7e4b173c80 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:53.236250 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:53.253778 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:53.326273 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde7143b-02f5-4378-b219-18e318173b65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:53.327083 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfe68ea-211c-4b59-b40f-dc7e4b173c80 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:53.327786 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde7143b-02f5-4378-b219-18e318173b65 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:53.389451 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc89dc89-552d-4ffd-9104-406a576add67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:53.390132 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89dc89-552d-4ffd-9104-406a576add67 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:53.390544 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89dc89-552d-4ffd-9104-406a576add67 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:53.391014 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89dc89-552d-4ffd-9104-406a576add67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:53.527809 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc89dc89-552d-4ffd-9104-406a576add67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:53.529310 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc89dc89-552d-4ffd-9104-406a576add67 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:53.568016 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 133/264] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:53:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 746 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 14:53:53.574478 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde7143b-02f5-4378-b219-18e318173b65 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:53.607155 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:53.672903 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 132/265] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:53:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 14:53:53.679893 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc89dc89-552d-4ffd-9104-406a576add67 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:53.726232 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:53.842684 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde7143b-02f5-4378-b219-18e318173b65 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:53.888946 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc89dc89-552d-4ffd-9104-406a576add67 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:53.932874 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2052d481-0bc9-4481-8c63-90cca1aadbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:53.933445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2052d481-0bc9-4481-8c63-90cca1aadbff None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:53.933724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2052d481-0bc9-4481-8c63-90cca1aadbff None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:53.933997 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2052d481-0bc9-4481-8c63-90cca1aadbff None admin] PATH_INFO: `/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:53.939548 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2052d481-0bc9-4481-8c63-90cca1aadbff None admin] RBAC: Authorizing `identity:get_project(project_id=8f8cdaafdd52411bb20efe51ddc2565c)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:53.941037 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2052d481-0bc9-4481-8c63-90cca1aadbff None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:53.943079 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 134/266] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:53:53 2022] GET /identity/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c => generated 302 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:53:53.948653 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2052d481-0bc9-4481-8c63-90cca1aadbff None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:53.979526 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7934ee04-0275-4aa8-9e21-e667a21236dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:53.980181 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934ee04-0275-4aa8-9e21-e667a21236dc None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:53.980463 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934ee04-0275-4aa8-9e21-e667a21236dc None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:53.980736 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934ee04-0275-4aa8-9e21-e667a21236dc None admin] PATH_INFO: `/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:53.981927 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7934ee04-0275-4aa8-9e21-e667a21236dc None admin] RBAC: Authorizing `identity:get_project(project_id=8f8cdaafdd52411bb20efe51ddc2565c)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:53.983433 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7934ee04-0275-4aa8-9e21-e667a21236dc None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:53.985214 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 133/267] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:53:53 2022] GET /identity/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c => generated 302 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:54.001327 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7934ee04-0275-4aa8-9e21-e667a21236dc None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:54.027361 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:54.190928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2052d481-0bc9-4481-8c63-90cca1aadbff None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:54.236578 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7934ee04-0275-4aa8-9e21-e667a21236dc None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:54.239084 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:54.254168 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc50890-74ed-419c-ac0b-7b7efa70f335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:54.254781 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc50890-74ed-419c-ac0b-7b7efa70f335 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:54.255077 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc50890-74ed-419c-ac0b-7b7efa70f335 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:54.255542 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc50890-74ed-419c-ac0b-7b7efa70f335 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:54.256585 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc50890-74ed-419c-ac0b-7b7efa70f335 None admin] RBAC: Adding query filter params (role.id=99e8fe7df8e7446680a81d4eaaa1cb68, scope.project.id=8f8cdaafdd52411bb20efe51ddc2565c, user.id=91b926a36a5f4024a2cd0788d2867305) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:54.257033 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc50890-74ed-419c-ac0b-7b7efa70f335 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:54.258619 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc50890-74ed-419c-ac0b-7b7efa70f335 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:54.265074 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 135/268] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:53:53 2022] GET /identity/v3/role_assignments?user.id=91b926a36a5f4024a2cd0788d2867305&scope.project.id=8f8cdaafdd52411bb20efe51ddc2565c&role.id=99e8fe7df8e7446680a81d4eaaa1cb68 => generated 608 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:53:54.266447 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3179806-5583-456f-b472-0e020e179786 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:54.266913 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3179806-5583-456f-b472-0e020e179786 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:54.267326 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3179806-5583-456f-b472-0e020e179786 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:54.268926 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 136/269] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:54.274629 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00cef4-9710-4c25-82b3-df2e7aa8bba9 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:54.275191 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00cef4-9710-4c25-82b3-df2e7aa8bba9 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:54.275604 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00cef4-9710-4c25-82b3-df2e7aa8bba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:54.304288 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-ea00cef4-9710-4c25-82b3-df2e7aa8bba9 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:54.305908 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-ea00cef4-9710-4c25-82b3-df2e7aa8bba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:54.326883 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de3555-bfca-47f8-80cd-b4b9730848fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:54.327776 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de3555-bfca-47f8-80cd-b4b9730848fb None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:54.328223 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de3555-bfca-47f8-80cd-b4b9730848fb None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:54.328665 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de3555-bfca-47f8-80cd-b4b9730848fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:54.329541 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de3555-bfca-47f8-80cd-b4b9730848fb None admin] RBAC: Adding query filter params (role.id=d1818eb7f2bd467fbf75bad5cc8e0aea, scope.project.id=8f8cdaafdd52411bb20efe51ddc2565c, user.id=91b926a36a5f4024a2cd0788d2867305) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:54.329988 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de3555-bfca-47f8-80cd-b4b9730848fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:54.331636 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de3555-bfca-47f8-80cd-b4b9730848fb None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:54.345784 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 134/270] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:53:53 2022] GET /identity/v3/role_assignments?user.id=91b926a36a5f4024a2cd0788d2867305&scope.project.id=8f8cdaafdd52411bb20efe51ddc2565c&role.id=d1818eb7f2bd467fbf75bad5cc8e0aea => generated 608 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:53:54.412432 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 137/271] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:54.559630 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e48f4-f932-44d4-a140-c91075b5c9df None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:54.560490 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e48f4-f932-44d4-a140-c91075b5c9df None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:54.560944 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e48f4-f932-44d4-a140-c91075b5c9df None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:54.562508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 135/272] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:54.572695 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ca26e-7dad-4ace-82b7-eaae6321426c None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:54.573604 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ca26e-7dad-4ace-82b7-eaae6321426c None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:54.574121 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ca26e-7dad-4ace-82b7-eaae6321426c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:54.603335 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-a44ca26e-7dad-4ace-82b7-eaae6321426c None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:54.604836 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a44ca26e-7dad-4ace-82b7-eaae6321426c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:54.631247 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb590a-9e00-40dd-b40c-06a4ab1124fe None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:54.631814 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb590a-9e00-40dd-b40c-06a4ab1124fe None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:54.632260 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb590a-9e00-40dd-b40c-06a4ab1124fe None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:54.634117 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 136/273] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:54.642842 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89396367-4a10-4d39-ac4b-88640fd76212 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:54.642842 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89396367-4a10-4d39-ac4b-88640fd76212 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:54.642842 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89396367-4a10-4d39-ac4b-88640fd76212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:54.661858 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-89396367-4a10-4d39-ac4b-88640fd76212 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:54.663643 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-89396367-4a10-4d39-ac4b-88640fd76212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:54.697714 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 138/274] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:53:54.760860 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 137/275] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:53:54.862832 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ca26e-7dad-4ace-82b7-eaae6321426c None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:54.924436 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:54.988340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ca26e-7dad-4ace-82b7-eaae6321426c None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:55.012002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89396367-4a10-4d39-ac4b-88640fd76212 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:55.015744 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:55.044268 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75455771-e999-4de7-ac2b-511be921ee13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:55.044937 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75455771-e999-4de7-ac2b-511be921ee13 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:55.045209 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75455771-e999-4de7-ac2b-511be921ee13 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:55.045481 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75455771-e999-4de7-ac2b-511be921ee13 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:55.067309 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75455771-e999-4de7-ac2b-511be921ee13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:53:55.072259 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-75455771-e999-4de7-ac2b-511be921ee13 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:53:55.072259 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:53:55.072259 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:53:55.072259 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:53:55.072259 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:53:55.072259 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:53:55.072259 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:53:55.072259 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:53:55.072259 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:53:55.072259 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:53:55.072259 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:53:55.072259 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:53:55.072259 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:53:55.072259 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:53:55.072259 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:53:55.072259 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:53:55.073162 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:53:55.073162 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:53:55.073162 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:53:55.073162 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:53:55.073162 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:53:55.073162 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:53:55.073162 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:53:55.073162 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:53:55.073162 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:53:55.073162 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:53:55.073162 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:53:55.073162 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:53:55.073162 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:53:55.073162 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:53:55.073162 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:53:55.073162 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:53:55.076356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:53:55.076356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:53:55.076356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:53:55.076356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:53:55.076356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:53:55.076356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:53:55.076356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:53:55.076356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:53:55.076356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:53:55.076356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:53:55.076356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:53:55.076356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:53:55.076356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 139/276] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:53:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:53:55.079882 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75455771-e999-4de7-ac2b-511be921ee13 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:55.152255 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89396367-4a10-4d39-ac4b-88640fd76212 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:55.187412 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75455771-e999-4de7-ac2b-511be921ee13 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:55.194565 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14320bb-abb5-4979-88ae-1a2acb1cc185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:55.195227 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14320bb-abb5-4979-88ae-1a2acb1cc185 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:55.195507 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14320bb-abb5-4979-88ae-1a2acb1cc185 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:55.195779 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14320bb-abb5-4979-88ae-1a2acb1cc185 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:55.200840 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:55.206217 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14320bb-abb5-4979-88ae-1a2acb1cc185 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:53:55.211107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-c14320bb-abb5-4979-88ae-1a2acb1cc185 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:53:55.211107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:53:55.211107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:53:55.211107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:53:55.211107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:53:55.211107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:53:55.211107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:53:55.211107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:53:55.211107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:53:55.211107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:53:55.211107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:53:55.211107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:53:55.211107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:53:55.211107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:53:55.211107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:53:55.211107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:53:55.212001 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:53:55.212001 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:53:55.212001 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:53:55.212001 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:53:55.212001 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:53:55.212001 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:53:55.212001 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:53:55.212001 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:53:55.212001 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:53:55.212001 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:53:55.212001 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:53:55.212001 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:53:55.212001 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:53:55.212001 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:53:55.212001 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:53:55.212001 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:53:55.213383 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:53:55.213383 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:53:55.213383 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:53:55.213383 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:53:55.213383 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:53:55.213383 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:53:55.213383 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:53:55.213383 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:53:55.213383 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:53:55.213383 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:53:55.213383 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:53:55.213383 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:53:55.213383 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 138/277] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:53:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:53:55.213383 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14320bb-abb5-4979-88ae-1a2acb1cc185 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:55.225343 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38dfeb-ddec-406c-9713-ec269c1376e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:55.225957 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38dfeb-ddec-406c-9713-ec269c1376e0 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:55.226236 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38dfeb-ddec-406c-9713-ec269c1376e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:55.226508 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38dfeb-ddec-406c-9713-ec269c1376e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:55.227213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f38dfeb-ddec-406c-9713-ec269c1376e0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:55.227514 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f38dfeb-ddec-406c-9713-ec269c1376e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:55.228906 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f38dfeb-ddec-406c-9713-ec269c1376e0 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:55.237938 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 140/278] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 14:53:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:55.238975 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee07d3f-7cf4-4e72-b9aa-905569c0f915 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:55.239259 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee07d3f-7cf4-4e72-b9aa-905569c0f915 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:55.239533 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee07d3f-7cf4-4e72-b9aa-905569c0f915 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:55.240729 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 141/279] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:55.241422 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee07d3f-7cf4-4e72-b9aa-905569c0f915 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:55.281690 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:55.320129 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14320bb-abb5-4979-88ae-1a2acb1cc185 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:55.343307 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee07d3f-7cf4-4e72-b9aa-905569c0f915 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:55.354060 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cbab46-610a-4511-9724-5b2aa6af8a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:55.354687 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbab46-610a-4511-9724-5b2aa6af8a63 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:55.354993 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbab46-610a-4511-9724-5b2aa6af8a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:55.355268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbab46-610a-4511-9724-5b2aa6af8a63 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:55.356374 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cbab46-610a-4511-9724-5b2aa6af8a63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:55.357824 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cbab46-610a-4511-9724-5b2aa6af8a63 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:55.359438 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 139/280] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:53:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:55.360336 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cbab46-610a-4511-9724-5b2aa6af8a63 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:55.381412 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1fcfe-5bcb-4403-aa06-23634b9d052d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:55.382099 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1fcfe-5bcb-4403-aa06-23634b9d052d None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:55.382470 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1fcfe-5bcb-4403-aa06-23634b9d052d None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:55.382827 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1fcfe-5bcb-4403-aa06-23634b9d052d None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:55.383575 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f1fcfe-5bcb-4403-aa06-23634b9d052d None admin] RBAC: Adding query filter params (name=member) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:55.383946 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f1fcfe-5bcb-4403-aa06-23634b9d052d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:55.385410 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f1fcfe-5bcb-4403-aa06-23634b9d052d None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:55.394785 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 142/281] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 14:53:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:55.396022 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6813723-1870-4a9b-b1aa-eb5ee1e9729f None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:55.396311 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6813723-1870-4a9b-b1aa-eb5ee1e9729f None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:55.396585 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6813723-1870-4a9b-b1aa-eb5ee1e9729f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:55.414272 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-b6813723-1870-4a9b-b1aa-eb5ee1e9729f None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:55.415410 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-b6813723-1870-4a9b-b1aa-eb5ee1e9729f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:55.444306 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:55.460856 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cbab46-610a-4511-9724-5b2aa6af8a63 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:55.481832 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 143/282] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:55.482967 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6813723-1870-4a9b-b1aa-eb5ee1e9729f None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:55.501784 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:55.508727 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c643a-977a-4ab5-a117-ab18d6349acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:55.509467 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c643a-977a-4ab5-a117-ab18d6349acd None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:55.509815 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c643a-977a-4ab5-a117-ab18d6349acd None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:55.510180 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c643a-977a-4ab5-a117-ab18d6349acd None admin] PATH_INFO: `/v3/users/aa385023402746d5ad582a05e3039347` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:55.511398 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47c643a-977a-4ab5-a117-ab18d6349acd None admin] RBAC: Authorizing `identity:get_user(user_id=aa385023402746d5ad582a05e3039347)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:55.512925 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47c643a-977a-4ab5-a117-ab18d6349acd None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:55.514724 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 140/283] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:53:55 2022] GET /identity/v3/users/aa385023402746d5ad582a05e3039347 => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:55.515413 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c643a-977a-4ab5-a117-ab18d6349acd None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:55.595022 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6813723-1870-4a9b-b1aa-eb5ee1e9729f None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:55.619411 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c643a-977a-4ab5-a117-ab18d6349acd None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:55.629127 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063bf77a-43fc-42d0-af57-c425f7c227c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:55.629732 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bf77a-43fc-42d0-af57-c425f7c227c0 None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:55.630015 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bf77a-43fc-42d0-af57-c425f7c227c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:55.630288 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bf77a-43fc-42d0-af57-c425f7c227c0 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:55.631045 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063bf77a-43fc-42d0-af57-c425f7c227c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:55.632488 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063bf77a-43fc-42d0-af57-c425f7c227c0 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:55.656539 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 144/284] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 14:53:55 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:53:55.657670 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063bf77a-43fc-42d0-af57-c425f7c227c0 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:55.659245 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c0cbc-3759-4dba-951a-f750e6f89815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:55.660017 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c0cbc-3759-4dba-951a-f750e6f89815 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:55.660496 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c0cbc-3759-4dba-951a-f750e6f89815 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:55.660774 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c0cbc-3759-4dba-951a-f750e6f89815 None admin] PATH_INFO: `/v3/users/0939a77927c342c4a27d597415b7bf96` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:55.683916 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3c0cbc-3759-4dba-951a-f750e6f89815 None admin] RBAC: Authorizing `identity:get_user(user_id=0939a77927c342c4a27d597415b7bf96)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:55.685931 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3c0cbc-3759-4dba-951a-f750e6f89815 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:55.688800 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:55.694979 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 141/285] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:53:55 2022] GET /identity/v3/users/0939a77927c342c4a27d597415b7bf96 => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:55.696764 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c0cbc-3759-4dba-951a-f750e6f89815 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:55.773973 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:55.800658 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063bf77a-43fc-42d0-af57-c425f7c227c0 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:55.827812 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c0cbc-3759-4dba-951a-f750e6f89815 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:55.840015 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60113b10-1e58-4c3b-b816-8a78ddb0431e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:55.840311 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60113b10-1e58-4c3b-b816-8a78ddb0431e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:55.840583 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60113b10-1e58-4c3b-b816-8a78ddb0431e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:55.840868 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60113b10-1e58-4c3b-b816-8a78ddb0431e None admin] PATH_INFO: `/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:55.841994 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60113b10-1e58-4c3b-b816-8a78ddb0431e None admin] RBAC: Authorizing `identity:get_project(project_id=8f8cdaafdd52411bb20efe51ddc2565c)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:55.843487 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60113b10-1e58-4c3b-b816-8a78ddb0431e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:55.849642 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 145/286] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:53:55 2022] GET /identity/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:55.849642 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60113b10-1e58-4c3b-b816-8a78ddb0431e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:55.871877 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99db23d-4c37-41d7-bade-db0578142c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:55.872496 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99db23d-4c37-41d7-bade-db0578142c60 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:55.872774 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99db23d-4c37-41d7-bade-db0578142c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:55.873670 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99db23d-4c37-41d7-bade-db0578142c60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:55.873786 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99db23d-4c37-41d7-bade-db0578142c60 None admin] RBAC: Adding query filter params (role.id=c9da950c75da4939b8bd649c11015650, user.id=0939a77927c342c4a27d597415b7bf96, scope.system=all) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:55.874040 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99db23d-4c37-41d7-bade-db0578142c60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:55.875930 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99db23d-4c37-41d7-bade-db0578142c60 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:55.886982 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 142/287] 10.222.0.33 () {56 vars in 1308 bytes} [Sun Sep 11 14:53:55 2022] GET /identity/v3/role_assignments?user.id=0939a77927c342c4a27d597415b7bf96&scope.system=all&role.id=c9da950c75da4939b8bd649c11015650 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:55.886982 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99db23d-4c37-41d7-bade-db0578142c60 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:55.976662 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:55.992719 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60113b10-1e58-4c3b-b816-8a78ddb0431e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:56.038137 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99db23d-4c37-41d7-bade-db0578142c60 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:56.043972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6220b24-a2c8-4e47-8f82-a5190b5572b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:56.044620 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6220b24-a2c8-4e47-8f82-a5190b5572b5 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:56.044903 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6220b24-a2c8-4e47-8f82-a5190b5572b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:56.045178 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6220b24-a2c8-4e47-8f82-a5190b5572b5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:56.061042 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6220b24-a2c8-4e47-8f82-a5190b5572b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:53:56.066851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-d6220b24-a2c8-4e47-8f82-a5190b5572b5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:53:56.066851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:53:56.066851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:53:56.066851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:53:56.066851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:53:56.066851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:53:56.066851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:53:56.066851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:53:56.066851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:53:56.066851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:53:56.066851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:53:56.066851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:53:56.066851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:53:56.066851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:53:56.066851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:53:56.066851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:53:56.067832 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:53:56.067832 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:53:56.067832 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:53:56.067832 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:53:56.067832 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:53:56.067832 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:53:56.067832 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:53:56.067832 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:53:56.067832 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:53:56.067832 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:53:56.067832 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:53:56.067832 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:53:56.067832 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:53:56.067832 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:53:56.067832 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:53:56.067832 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:53:56.068619 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:53:56.068619 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:53:56.068619 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:53:56.068619 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:53:56.068619 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:53:56.068619 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:53:56.068619 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:53:56.068619 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:53:56.068619 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:53:56.068619 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:53:56.068619 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:53:56.068619 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:53:56.068619 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 146/288] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:53:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:53:56.075806 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6220b24-a2c8-4e47-8f82-a5190b5572b5 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:56.089109 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e48a9b-c29c-4c66-877c-dbbb742a4507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:56.090379 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e48a9b-c29c-4c66-877c-dbbb742a4507 None admin] REQUEST_METHOD: `PUT` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:56.091015 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e48a9b-c29c-4c66-877c-dbbb742a4507 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:56.091576 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e48a9b-c29c-4c66-877c-dbbb742a4507 None admin] PATH_INFO: `/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c/users/aa385023402746d5ad582a05e3039347/roles/06e98804662441c88b8871c32b9468aa` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:56.095108 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e48a9b-c29c-4c66-877c-dbbb742a4507 None admin] RBAC: Authorizing `identity:create_grant(project_id=8f8cdaafdd52411bb20efe51ddc2565c, user_id=aa385023402746d5ad582a05e3039347, role_id=06e98804662441c88b8871c32b9468aa)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:56.098086 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e48a9b-c29c-4c66-877c-dbbb742a4507 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:56.103170 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:56.123796 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 143/289] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 14:53:55 2022] PUT /identity/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c/users/aa385023402746d5ad582a05e3039347/roles/06e98804662441c88b8871c32b9468aa => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:53:56.199021 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6220b24-a2c8-4e47-8f82-a5190b5572b5 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:56.239029 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc980a-b777-4c5b-a9de-2aa0f4372340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:56.239576 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc980a-b777-4c5b-a9de-2aa0f4372340 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:56.239861 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc980a-b777-4c5b-a9de-2aa0f4372340 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:56.240134 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc980a-b777-4c5b-a9de-2aa0f4372340 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:56.240899 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc980a-b777-4c5b-a9de-2aa0f4372340 None admin] RBAC: Adding query filter params (name=member) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:56.241205 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc980a-b777-4c5b-a9de-2aa0f4372340 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:56.242644 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc980a-b777-4c5b-a9de-2aa0f4372340 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:56.253401 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 147/290] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 14:53:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:53:56.259255 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e48a9b-c29c-4c66-877c-dbbb742a4507 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:56.374275 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e48a9b-c29c-4c66-877c-dbbb742a4507 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:56.414997 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e203bc0d-4d19-4db1-9d6c-551d007f6f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:56.415642 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e203bc0d-4d19-4db1-9d6c-551d007f6f32 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:56.415924 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e203bc0d-4d19-4db1-9d6c-551d007f6f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:56.416200 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e203bc0d-4d19-4db1-9d6c-551d007f6f32 None admin] PATH_INFO: `/v3/users/6a624b32bd1945c883b0b5be93482487` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:56.417346 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e203bc0d-4d19-4db1-9d6c-551d007f6f32 None admin] RBAC: Authorizing `identity:get_user(user_id=6a624b32bd1945c883b0b5be93482487)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:56.418854 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e203bc0d-4d19-4db1-9d6c-551d007f6f32 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:56.420670 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 144/291] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:53:56 2022] GET /identity/v3/users/6a624b32bd1945c883b0b5be93482487 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:56.426323 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc980a-b777-4c5b-a9de-2aa0f4372340 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:56.444657 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:56.559715 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc980a-b777-4c5b-a9de-2aa0f4372340 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:56.596599 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf95c65-dabe-4902-83be-f76101c3d599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:56.597222 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf95c65-dabe-4902-83be-f76101c3d599 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:56.597476 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf95c65-dabe-4902-83be-f76101c3d599 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:56.597749 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf95c65-dabe-4902-83be-f76101c3d599 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:56.669523 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf95c65-dabe-4902-83be-f76101c3d599 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:56.671033 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf95c65-dabe-4902-83be-f76101c3d599 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:56.695188 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:56.778972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 148/292] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:53:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:53:56.784450 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e203bc0d-4d19-4db1-9d6c-551d007f6f32 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:56.864642 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:56.921493 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e203bc0d-4d19-4db1-9d6c-551d007f6f32 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:56.960445 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f540fe6-a662-4137-bb5e-6de0a79e45ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:56.961427 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f540fe6-a662-4137-bb5e-6de0a79e45ab None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:56.961922 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f540fe6-a662-4137-bb5e-6de0a79e45ab None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:56.962388 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f540fe6-a662-4137-bb5e-6de0a79e45ab None admin] PATH_INFO: `/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:56.963781 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f540fe6-a662-4137-bb5e-6de0a79e45ab None admin] RBAC: Authorizing `identity:get_project(project_id=8f8cdaafdd52411bb20efe51ddc2565c)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:56.965446 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f540fe6-a662-4137-bb5e-6de0a79e45ab None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:56.967637 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 145/293] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:53:56 2022] GET /identity/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:56.974829 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf95c65-dabe-4902-83be-f76101c3d599 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:57.098943 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf95c65-dabe-4902-83be-f76101c3d599 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:57.112698 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:57.141746 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f147c-0908-49e2-b59f-8b01a4e97b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:57.142281 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f147c-0908-49e2-b59f-8b01a4e97b67 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:57.142562 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f147c-0908-49e2-b59f-8b01a4e97b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:57.142851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f147c-0908-49e2-b59f-8b01a4e97b67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:57.143585 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33f147c-0908-49e2-b59f-8b01a4e97b67 None admin] RBAC: Adding query filter params (role.id=c9da950c75da4939b8bd649c11015650, scope.project.id=8f8cdaafdd52411bb20efe51ddc2565c, user.id=6a624b32bd1945c883b0b5be93482487) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:57.143893 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33f147c-0908-49e2-b59f-8b01a4e97b67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:57.145353 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33f147c-0908-49e2-b59f-8b01a4e97b67 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:57.154355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 149/294] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:53:56 2022] GET /identity/v3/role_assignments?user.id=6a624b32bd1945c883b0b5be93482487&scope.project.id=8f8cdaafdd52411bb20efe51ddc2565c&role.id=c9da950c75da4939b8bd649c11015650 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:58.263133 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565aad1e-40e8-472d-8b32-5d6a0e2cc173 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:58.263639 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565aad1e-40e8-472d-8b32-5d6a0e2cc173 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:58.263639 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565aad1e-40e8-472d-8b32-5d6a0e2cc173 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:58.265284 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 146/295] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:58.271538 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8ae43-b11b-4a62-8daa-0e6f256c44ed None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:58.272096 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8ae43-b11b-4a62-8daa-0e6f256c44ed None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:58.272096 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8ae43-b11b-4a62-8daa-0e6f256c44ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:58.291889 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-d3b8ae43-b11b-4a62-8daa-0e6f256c44ed None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:58.293112 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-d3b8ae43-b11b-4a62-8daa-0e6f256c44ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:58.370142 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 150/296] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:58.401602 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a938e-b3f8-432d-b56b-c2a0d9fcdd51 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:58.401921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a938e-b3f8-432d-b56b-c2a0d9fcdd51 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:58.402222 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a938e-b3f8-432d-b56b-c2a0d9fcdd51 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:58.403876 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 147/297] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:58.409115 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70229930-6859-4193-aeb9-7ae1d8f5169e None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:58.409411 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70229930-6859-4193-aeb9-7ae1d8f5169e None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:58.409687 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70229930-6859-4193-aeb9-7ae1d8f5169e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:58.428471 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-70229930-6859-4193-aeb9-7ae1d8f5169e None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:58.429672 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-70229930-6859-4193-aeb9-7ae1d8f5169e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:58.501830 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 151/298] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:58.648364 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12816a-c6ee-4835-a80c-5a7af412b680 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:58.648988 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12816a-c6ee-4835-a80c-5a7af412b680 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:58.649504 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12816a-c6ee-4835-a80c-5a7af412b680 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:58.651341 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 148/299] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:58.653021 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70229930-6859-4193-aeb9-7ae1d8f5169e None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:58.657523 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41ac08-be52-4b5e-9249-6c0944099421 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:58.657994 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41ac08-be52-4b5e-9249-6c0944099421 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:58.658344 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41ac08-be52-4b5e-9249-6c0944099421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:58.677101 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-fb41ac08-be52-4b5e-9249-6c0944099421 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:58.678532 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-fb41ac08-be52-4b5e-9249-6c0944099421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:58.694506 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:58.765079 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 149/300] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:53:58.775787 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb41ac08-be52-4b5e-9249-6c0944099421 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:58.793994 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70229930-6859-4193-aeb9-7ae1d8f5169e None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:58.830256 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c0fd6-28f1-46be-9e6d-2cb00b7e7a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:58.830926 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c0fd6-28f1-46be-9e6d-2cb00b7e7a79 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:58.831210 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c0fd6-28f1-46be-9e6d-2cb00b7e7a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:58.831485 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c0fd6-28f1-46be-9e6d-2cb00b7e7a79 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:58.837807 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:58.842252 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69c0fd6-28f1-46be-9e6d-2cb00b7e7a79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:53:58.847882 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-b69c0fd6-28f1-46be-9e6d-2cb00b7e7a79 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:53:58.847882 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:53:58.847882 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:53:58.847882 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:53:58.847882 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:53:58.847882 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:53:58.847882 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:53:58.847882 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:53:58.847882 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:53:58.847882 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:53:58.847882 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:53:58.847882 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:53:58.847882 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:53:58.847882 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:53:58.847882 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:53:58.847882 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:53:58.848806 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:53:58.848806 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:53:58.848806 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:53:58.848806 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:53:58.848806 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:53:58.848806 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:53:58.848806 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:53:58.848806 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:53:58.848806 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:53:58.848806 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:53:58.848806 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:53:58.848806 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:53:58.848806 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:53:58.848806 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:53:58.848806 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:53:58.848806 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:53:58.849603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:53:58.849603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:53:58.849603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:53:58.849603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:53:58.849603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:53:58.849603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:53:58.849603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:53:58.849603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:53:58.849603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:53:58.849603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:53:58.849603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:53:58.849603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:53:58.849603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 152/301] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:53:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:53:58.850254 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62289be6-c5e5-482d-aaab-b68c957962f2 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:58.850854 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62289be6-c5e5-482d-aaab-b68c957962f2 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:58.850854 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62289be6-c5e5-482d-aaab-b68c957962f2 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:58.852073 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 153/302] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:53:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:53:58.855017 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62289be6-c5e5-482d-aaab-b68c957962f2 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:58.903182 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb41ac08-be52-4b5e-9249-6c0944099421 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:58.941134 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f157bbd4-115e-47f5-a68f-cdabc0c6d638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:58.941763 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157bbd4-115e-47f5-a68f-cdabc0c6d638 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:58.942040 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157bbd4-115e-47f5-a68f-cdabc0c6d638 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:58.942310 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157bbd4-115e-47f5-a68f-cdabc0c6d638 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:58.952867 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f157bbd4-115e-47f5-a68f-cdabc0c6d638 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:53:58.956018 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:58.959184 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-f157bbd4-115e-47f5-a68f-cdabc0c6d638 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:53:58.959184 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:53:58.959184 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:53:58.959184 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:53:58.959184 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:53:58.959184 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:53:58.959184 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:53:58.959184 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:53:58.959184 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:53:58.959184 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:53:58.959184 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:53:58.959184 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:53:58.959184 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:53:58.959184 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:53:58.959184 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:53:58.959184 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:53:58.960149 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:53:58.960149 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:53:58.960149 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:53:58.960149 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:53:58.960149 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:53:58.960149 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:53:58.960149 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:53:58.960149 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:53:58.960149 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:53:58.960149 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:53:58.960149 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:53:58.960149 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:53:58.960149 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:53:58.960149 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:53:58.960149 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:53:58.960149 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:53:58.960933 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:53:58.960933 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:53:58.960933 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:53:58.960933 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:53:58.960933 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:53:58.960933 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:53:58.960933 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:53:58.960933 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:53:58.960933 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:53:58.960933 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:53:58.960933 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:53:58.960933 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:53:58.960933 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 150/303] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:53:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:53:58.961587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b67734-e5eb-403f-b8d4-2f6e3f990df6 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:58.961587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b67734-e5eb-403f-b8d4-2f6e3f990df6 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:58.962051 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b67734-e5eb-403f-b8d4-2f6e3f990df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:58.968764 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62289be6-c5e5-482d-aaab-b68c957962f2 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:58.980984 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-b9b67734-e5eb-403f-b8d4-2f6e3f990df6 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:53:58.982171 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-b9b67734-e5eb-403f-b8d4-2f6e3f990df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:53:59.006751 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c530e6-1d96-4d14-b7e7-6030fdc6b9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:59.007376 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c530e6-1d96-4d14-b7e7-6030fdc6b9f6 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:59.007744 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c530e6-1d96-4d14-b7e7-6030fdc6b9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:59.008087 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c530e6-1d96-4d14-b7e7-6030fdc6b9f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:59.008788 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c530e6-1d96-4d14-b7e7-6030fdc6b9f6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:59.009085 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c530e6-1d96-4d14-b7e7-6030fdc6b9f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:59.010520 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c530e6-1d96-4d14-b7e7-6030fdc6b9f6 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:59.019995 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 154/304] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 14:53:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:53:59.020608 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c530e6-1d96-4d14-b7e7-6030fdc6b9f6 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:59.060147 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 151/305] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:53:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:53:59.061004 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b67734-e5eb-403f-b8d4-2f6e3f990df6 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:59.099443 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:59.123746 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c530e6-1d96-4d14-b7e7-6030fdc6b9f6 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:59.160042 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84340dc3-b715-4b9f-8815-e2bb8f8c9158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:59.160588 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84340dc3-b715-4b9f-8815-e2bb8f8c9158 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:59.160856 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84340dc3-b715-4b9f-8815-e2bb8f8c9158 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:59.161132 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84340dc3-b715-4b9f-8815-e2bb8f8c9158 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:59.161792 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84340dc3-b715-4b9f-8815-e2bb8f8c9158 None admin] RBAC: Adding query filter params (name=member) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:59.162094 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84340dc3-b715-4b9f-8815-e2bb8f8c9158 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:59.163501 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84340dc3-b715-4b9f-8815-e2bb8f8c9158 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:59.171515 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b67734-e5eb-403f-b8d4-2f6e3f990df6 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:59.171750 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 155/306] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 14:53:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:59.177678 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84340dc3-b715-4b9f-8815-e2bb8f8c9158 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:59.200340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:59.209834 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1524ea-69b0-4353-8f46-700010fa7e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:59.210523 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1524ea-69b0-4353-8f46-700010fa7e7d None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:59.210987 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1524ea-69b0-4353-8f46-700010fa7e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:59.211315 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1524ea-69b0-4353-8f46-700010fa7e7d None admin] PATH_INFO: `/v3/users/4f1b2b9a76494f4789a10a53fc478b44` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:59.226009 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1524ea-69b0-4353-8f46-700010fa7e7d None admin] RBAC: Authorizing `identity:get_user(user_id=4f1b2b9a76494f4789a10a53fc478b44)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:59.227547 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1524ea-69b0-4353-8f46-700010fa7e7d None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:59.229157 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 152/307] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:53:59 2022] GET /identity/v3/users/4f1b2b9a76494f4789a10a53fc478b44 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:59.229908 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1524ea-69b0-4353-8f46-700010fa7e7d None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:59.287266 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84340dc3-b715-4b9f-8815-e2bb8f8c9158 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:59.322420 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6d4f0c-b330-49ac-9002-0fef2a463474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:59.323368 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d4f0c-b330-49ac-9002-0fef2a463474 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:59.323653 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d4f0c-b330-49ac-9002-0fef2a463474 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:59.323924 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d4f0c-b330-49ac-9002-0fef2a463474 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:59.325034 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6d4f0c-b330-49ac-9002-0fef2a463474 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:59.327027 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6d4f0c-b330-49ac-9002-0fef2a463474 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:59.328676 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 156/308] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:53:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:59.329442 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6d4f0c-b330-49ac-9002-0fef2a463474 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:59.335643 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:59.346374 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1524ea-69b0-4353-8f46-700010fa7e7d None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:59.383225 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda3239-25c0-44b1-822c-48a19c06bcd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:59.383862 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda3239-25c0-44b1-822c-48a19c06bcd5 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:59.384145 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda3239-25c0-44b1-822c-48a19c06bcd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:59.384419 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda3239-25c0-44b1-822c-48a19c06bcd5 None admin] PATH_INFO: `/v3/users/0939a77927c342c4a27d597415b7bf96` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:59.385547 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feda3239-25c0-44b1-822c-48a19c06bcd5 None admin] RBAC: Authorizing `identity:get_user(user_id=0939a77927c342c4a27d597415b7bf96)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:59.387024 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feda3239-25c0-44b1-822c-48a19c06bcd5 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:59.388612 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 153/309] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:53:59 2022] GET /identity/v3/users/0939a77927c342c4a27d597415b7bf96 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:59.389344 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda3239-25c0-44b1-822c-48a19c06bcd5 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:59.423067 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:59.436380 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6d4f0c-b330-49ac-9002-0fef2a463474 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:59.470811 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90f78c-1db1-4aa5-a7a8-76ea91a9a33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:59.471448 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90f78c-1db1-4aa5-a7a8-76ea91a9a33c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:59.471728 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90f78c-1db1-4aa5-a7a8-76ea91a9a33c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:59.472007 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90f78c-1db1-4aa5-a7a8-76ea91a9a33c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:59.472727 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe90f78c-1db1-4aa5-a7a8-76ea91a9a33c None admin] RBAC: Adding query filter params (role.id=06e98804662441c88b8871c32b9468aa, user.id=4f1b2b9a76494f4789a10a53fc478b44, scope.system=all) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:59.473035 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe90f78c-1db1-4aa5-a7a8-76ea91a9a33c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:59.474490 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe90f78c-1db1-4aa5-a7a8-76ea91a9a33c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:59.479320 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 157/310] 10.222.0.33 () {56 vars in 1308 bytes} [Sun Sep 11 14:53:59 2022] GET /identity/v3/role_assignments?user.id=4f1b2b9a76494f4789a10a53fc478b44&scope.system=all&role.id=06e98804662441c88b8871c32b9468aa => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:53:59.480098 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90f78c-1db1-4aa5-a7a8-76ea91a9a33c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:59.492399 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda3239-25c0-44b1-822c-48a19c06bcd5 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:59.532830 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb6ce4c-f0f0-47b0-a66b-2c7bf62fff23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:59.533470 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6ce4c-f0f0-47b0-a66b-2c7bf62fff23 None admin] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:59.533759 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6ce4c-f0f0-47b0-a66b-2c7bf62fff23 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:59.534035 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6ce4c-f0f0-47b0-a66b-2c7bf62fff23 None admin] PATH_INFO: `/v3/groups` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:59.534690 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb6ce4c-f0f0-47b0-a66b-2c7bf62fff23 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:59.536240 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb6ce4c-f0f0-47b0-a66b-2c7bf62fff23 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:59.557256 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 154/311] 10.222.0.33 () {60 vars in 1134 bytes} [Sun Sep 11 14:53:59 2022] POST /identity/v3/groups => generated 229 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:53:59.558368 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb6ce4c-f0f0-47b0-a66b-2c7bf62fff23 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:59.604095 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:59.623374 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90f78c-1db1-4aa5-a7a8-76ea91a9a33c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:59.677231 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb85092-e390-463e-a494-d2d419a9a4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:59.677812 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb85092-e390-463e-a494-d2d419a9a4ea None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:59.678097 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb85092-e390-463e-a494-d2d419a9a4ea None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:59.678369 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb85092-e390-463e-a494-d2d419a9a4ea None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:59.691694 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb85092-e390-463e-a494-d2d419a9a4ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:53:59.697730 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-2bb85092-e390-463e-a494-d2d419a9a4ea None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:53:59.697730 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:53:59.697730 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:53:59.697730 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:53:59.697730 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:53:59.697730 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:53:59.697730 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:53:59.697730 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:53:59.697730 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:53:59.697730 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:53:59.697730 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:53:59.697730 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:53:59.697730 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:53:59.697730 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:53:59.697730 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:53:59.697730 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:53:59.698686 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:53:59.698686 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:53:59.698686 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:53:59.698686 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:53:59.698686 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:53:59.698686 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:53:59.698686 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:53:59.698686 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:53:59.698686 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:53:59.698686 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:53:59.698686 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:53:59.698686 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:53:59.698686 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:53:59.698686 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:53:59.698686 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:53:59.698686 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:53:59.699535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:53:59.699535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:53:59.699535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:53:59.699535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:53:59.699535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:53:59.699535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:53:59.699535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:53:59.699535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:53:59.699535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:53:59.699535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:53:59.699535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:53:59.699535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:53:59.699535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 158/312] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:53:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:53:59.705963 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb85092-e390-463e-a494-d2d419a9a4ea None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:53:59.723721 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb6ce4c-f0f0-47b0-a66b-2c7bf62fff23 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:59.727314 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:53:59.770467 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191f78a-f88f-4c97-9f1e-3a8fc6f6f8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:59.771278 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191f78a-f88f-4c97-9f1e-3a8fc6f6f8a3 None admin] REQUEST_METHOD: `PUT` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:59.771562 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191f78a-f88f-4c97-9f1e-3a8fc6f6f8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:59.771842 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191f78a-f88f-4c97-9f1e-3a8fc6f6f8a3 None admin] PATH_INFO: `/v3/system/users/0939a77927c342c4a27d597415b7bf96/roles/c9da950c75da4939b8bd649c11015650` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:59.773724 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6191f78a-f88f-4c97-9f1e-3a8fc6f6f8a3 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0939a77927c342c4a27d597415b7bf96, role_id=c9da950c75da4939b8bd649c11015650)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:59.775962 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6191f78a-f88f-4c97-9f1e-3a8fc6f6f8a3 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:59.786236 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 155/313] 10.222.0.33 () {58 vars in 1334 bytes} [Sun Sep 11 14:53:59 2022] PUT /identity/v3/system/users/0939a77927c342c4a27d597415b7bf96/roles/c9da950c75da4939b8bd649c11015650 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:53:59.853314 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb85092-e390-463e-a494-d2d419a9a4ea None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:53:59.895499 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8647a4-8ac0-413e-aad7-25ba940b3b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:53:59.896227 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8647a4-8ac0-413e-aad7-25ba940b3b12 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:53:59.896583 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8647a4-8ac0-413e-aad7-25ba940b3b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:53:59.896931 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8647a4-8ac0-413e-aad7-25ba940b3b12 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:53:59.897698 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8647a4-8ac0-413e-aad7-25ba940b3b12 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:53:59.898075 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8647a4-8ac0-413e-aad7-25ba940b3b12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:53:59.899632 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8647a4-8ac0-413e-aad7-25ba940b3b12 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:53:59.911197 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 159/314] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 14:53:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:53:59.917126 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191f78a-f88f-4c97-9f1e-3a8fc6f6f8a3 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:00.023248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:00.039948 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191f78a-f88f-4c97-9f1e-3a8fc6f6f8a3 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:00.078943 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946a8b1-52ea-4542-b8bc-138fc19e5c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:00.079865 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946a8b1-52ea-4542-b8bc-138fc19e5c28 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:00.080287 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946a8b1-52ea-4542-b8bc-138fc19e5c28 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:00.080701 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946a8b1-52ea-4542-b8bc-138fc19e5c28 None admin] PATH_INFO: `/v3/users/aa385023402746d5ad582a05e3039347` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:00.082025 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c946a8b1-52ea-4542-b8bc-138fc19e5c28 None admin] RBAC: Authorizing `identity:get_user(user_id=aa385023402746d5ad582a05e3039347)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:00.083773 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c946a8b1-52ea-4542-b8bc-138fc19e5c28 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:00.085886 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 156/315] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:53:59 2022] GET /identity/v3/users/aa385023402746d5ad582a05e3039347 => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:00.091328 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8647a4-8ac0-413e-aad7-25ba940b3b12 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:00.173400 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:00.201933 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8647a4-8ac0-413e-aad7-25ba940b3b12 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:00.239025 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7e51a1-1f48-4a09-a123-d6ab56db42ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:00.239642 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e51a1-1f48-4a09-a123-d6ab56db42ad None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:00.239922 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e51a1-1f48-4a09-a123-d6ab56db42ad None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:00.240197 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e51a1-1f48-4a09-a123-d6ab56db42ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:00.315570 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7e51a1-1f48-4a09-a123-d6ab56db42ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:00.316995 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7e51a1-1f48-4a09-a123-d6ab56db42ad None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:00.396351 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:00.425669 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 160/316] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:54:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:54:00.431315 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946a8b1-52ea-4542-b8bc-138fc19e5c28 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:00.537632 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946a8b1-52ea-4542-b8bc-138fc19e5c28 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:00.575902 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc84d59f-ce5b-4b83-9ffb-969779eb4dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:00.576904 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84d59f-ce5b-4b83-9ffb-969779eb4dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:00.577383 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84d59f-ce5b-4b83-9ffb-969779eb4dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:00.577852 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84d59f-ce5b-4b83-9ffb-969779eb4dc2 None admin] PATH_INFO: `/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:00.579715 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc84d59f-ce5b-4b83-9ffb-969779eb4dc2 None admin] RBAC: Authorizing `identity:get_project(project_id=8f8cdaafdd52411bb20efe51ddc2565c)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:00.581400 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc84d59f-ce5b-4b83-9ffb-969779eb4dc2 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:00.583641 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 157/317] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:54:00 2022] GET /identity/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:00.594832 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7e51a1-1f48-4a09-a123-d6ab56db42ad None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:00.697904 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7e51a1-1f48-4a09-a123-d6ab56db42ad None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:00.740746 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b7dcd-267d-4075-a28f-2293dfc8da30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:00.741381 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b7dcd-267d-4075-a28f-2293dfc8da30 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:00.741662 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b7dcd-267d-4075-a28f-2293dfc8da30 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:00.741933 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b7dcd-267d-4075-a28f-2293dfc8da30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:00.742668 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b7dcd-267d-4075-a28f-2293dfc8da30 None admin] RBAC: Adding query filter params (role.id=06e98804662441c88b8871c32b9468aa, scope.project.id=8f8cdaafdd52411bb20efe51ddc2565c, user.id=aa385023402746d5ad582a05e3039347) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:00.743021 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b7dcd-267d-4075-a28f-2293dfc8da30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:00.744491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b7dcd-267d-4075-a28f-2293dfc8da30 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:00.752171 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 161/318] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:54:00 2022] GET /identity/v3/role_assignments?user.id=aa385023402746d5ad582a05e3039347&scope.project.id=8f8cdaafdd52411bb20efe51ddc2565c&role.id=06e98804662441c88b8871c32b9468aa => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:01.699424 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322f75b-02a6-4b1f-9794-6b44f349e79d None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:01.699946 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322f75b-02a6-4b1f-9794-6b44f349e79d None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:01.699946 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322f75b-02a6-4b1f-9794-6b44f349e79d None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:01.701442 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 158/319] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:01.706737 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd0914-e7de-4c7b-ac86-3641e2977eb8 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:01.707140 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd0914-e7de-4c7b-ac86-3641e2977eb8 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:01.707311 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd0914-e7de-4c7b-ac86-3641e2977eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:01.726173 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-99fd0914-e7de-4c7b-ac86-3641e2977eb8 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:01.727464 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-99fd0914-e7de-4c7b-ac86-3641e2977eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:01.758712 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:01.802844 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 162/320] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:01.977684 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fa8b0-8a41-40db-b48a-335032beaa27 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:01.977987 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fa8b0-8a41-40db-b48a-335032beaa27 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:01.978238 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fa8b0-8a41-40db-b48a-335032beaa27 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:01.979764 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 159/321] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:01.985496 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd770a30-4000-4aa7-af00-46150ff3f98a None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:01.985786 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd770a30-4000-4aa7-af00-46150ff3f98a None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:01.986046 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd770a30-4000-4aa7-af00-46150ff3f98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:02.005388 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-dd770a30-4000-4aa7-af00-46150ff3f98a None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:02.006759 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-dd770a30-4000-4aa7-af00-46150ff3f98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:02.026510 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85bb0d-8ecb-4fd6-9e5b-4c8db0d43e09 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:02.026793 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85bb0d-8ecb-4fd6-9e5b-4c8db0d43e09 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:02.027106 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85bb0d-8ecb-4fd6-9e5b-4c8db0d43e09 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:02.028573 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 160/322] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:54:02.033649 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198b26b-4aef-403a-82dd-a242a2779b00 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:02.034134 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198b26b-4aef-403a-82dd-a242a2779b00 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:02.034616 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198b26b-4aef-403a-82dd-a242a2779b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:02.053755 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-4198b26b-4aef-403a-82dd-a242a2779b00 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:02.055292 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-4198b26b-4aef-403a-82dd-a242a2779b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:02.073259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:02.080077 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 163/323] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:02.080924 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd770a30-4000-4aa7-af00-46150ff3f98a None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:02.127863 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 161/324] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:54:02.187084 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:02.191334 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd770a30-4000-4aa7-af00-46150ff3f98a None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:02.226367 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a799095-298a-4f2c-9e25-b3d43670c0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:02.227008 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a799095-298a-4f2c-9e25-b3d43670c0d0 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:02.227285 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a799095-298a-4f2c-9e25-b3d43670c0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:02.227555 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a799095-298a-4f2c-9e25-b3d43670c0d0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:02.237906 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a799095-298a-4f2c-9e25-b3d43670c0d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:02.242648 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-9a799095-298a-4f2c-9e25-b3d43670c0d0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:54:02.242648 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:02.242648 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:02.242648 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:02.242648 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:02.242648 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:02.242648 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:02.242648 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:02.242648 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:02.242648 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:02.242648 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:02.242648 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:02.242648 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:02.242648 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:02.242648 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:02.242648 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:02.243569 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:02.243569 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:02.243569 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:02.243569 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:02.243569 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:02.243569 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:02.243569 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:02.243569 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:02.243569 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:02.243569 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:02.243569 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:02.243569 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:02.243569 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:02.243569 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:02.243569 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:02.243569 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:02.247709 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:02.247709 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:02.247709 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:02.247709 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:02.247709 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:02.247709 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:02.247709 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:02.247709 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:02.247709 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:02.247709 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:02.247709 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:54:02.247709 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:02.247709 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 164/325] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:54:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:02.250584 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198b26b-4aef-403a-82dd-a242a2779b00 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:02.349381 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a799095-298a-4f2c-9e25-b3d43670c0d0 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:02.352540 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198b26b-4aef-403a-82dd-a242a2779b00 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:02.391411 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865bdfb7-20b8-4e0d-a2c4-7d2c083ee1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:02.392030 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865bdfb7-20b8-4e0d-a2c4-7d2c083ee1a5 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:02.392309 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865bdfb7-20b8-4e0d-a2c4-7d2c083ee1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:02.392578 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865bdfb7-20b8-4e0d-a2c4-7d2c083ee1a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:02.393239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865bdfb7-20b8-4e0d-a2c4-7d2c083ee1a5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:02.393538 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865bdfb7-20b8-4e0d-a2c4-7d2c083ee1a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:02.394951 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865bdfb7-20b8-4e0d-a2c4-7d2c083ee1a5 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:02.403237 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 162/326] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 14:54:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:02.404028 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865bdfb7-20b8-4e0d-a2c4-7d2c083ee1a5 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:02.419309 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:02.457459 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a799095-298a-4f2c-9e25-b3d43670c0d0 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:02.491727 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5f83d-dcb7-475c-a14c-95110ea047c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:02.492284 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5f83d-dcb7-475c-a14c-95110ea047c2 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:02.492633 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5f83d-dcb7-475c-a14c-95110ea047c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:02.492931 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5f83d-dcb7-475c-a14c-95110ea047c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:02.494049 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c5f83d-dcb7-475c-a14c-95110ea047c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:02.495555 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c5f83d-dcb7-475c-a14c-95110ea047c2 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:02.497124 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 165/327] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:54:02.500327 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5f83d-dcb7-475c-a14c-95110ea047c2 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:02.506084 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865bdfb7-20b8-4e0d-a2c4-7d2c083ee1a5 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:02.527353 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:02.540173 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024b6dc-86c2-4169-b6ce-a458ab6eef0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:02.540808 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024b6dc-86c2-4169-b6ce-a458ab6eef0c None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:02.541228 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024b6dc-86c2-4169-b6ce-a458ab6eef0c None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:02.541377 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024b6dc-86c2-4169-b6ce-a458ab6eef0c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:02.553061 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3024b6dc-86c2-4169-b6ce-a458ab6eef0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:02.558625 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-3024b6dc-86c2-4169-b6ce-a458ab6eef0c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:54:02.558625 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:02.558625 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:02.558625 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:02.558625 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:02.558625 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:02.558625 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:02.558625 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:02.558625 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:02.558625 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:02.558625 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:02.558625 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:02.558625 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:02.558625 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:02.558625 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:02.558625 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:02.559737 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:02.559737 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:02.559737 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:02.559737 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:02.559737 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:02.559737 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:02.559737 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:02.559737 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:02.559737 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:02.559737 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:02.559737 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:02.559737 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:02.559737 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:02.559737 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:02.559737 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:02.559737 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:02.560857 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:02.560857 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:02.560857 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:02.560857 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:02.560857 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:02.560857 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:02.560857 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:02.560857 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:02.560857 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:02.560857 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:02.560857 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:54:02.560857 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:02.560857 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 163/328] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:54:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:02.560857 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024b6dc-86c2-4169-b6ce-a458ab6eef0c None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:02.605523 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5f83d-dcb7-475c-a14c-95110ea047c2 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:02.635587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:02.640095 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e51f302-1f93-43ee-b445-59f9a9ca2a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:02.641231 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51f302-1f93-43ee-b445-59f9a9ca2a5d None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:02.641505 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51f302-1f93-43ee-b445-59f9a9ca2a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:02.641778 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51f302-1f93-43ee-b445-59f9a9ca2a5d None admin] PATH_INFO: `/v3/users/4f1b2b9a76494f4789a10a53fc478b44` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:02.642937 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e51f302-1f93-43ee-b445-59f9a9ca2a5d None admin] RBAC: Authorizing `identity:get_user(user_id=4f1b2b9a76494f4789a10a53fc478b44)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:02.644398 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e51f302-1f93-43ee-b445-59f9a9ca2a5d None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:02.645974 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 166/329] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:54:02 2022] GET /identity/v3/users/4f1b2b9a76494f4789a10a53fc478b44 => generated 293 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:02.646731 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e51f302-1f93-43ee-b445-59f9a9ca2a5d None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:02.662008 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024b6dc-86c2-4169-b6ce-a458ab6eef0c None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:02.695294 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da56ad1-a56c-4bf6-a05e-bc80eed5b2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:02.695901 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da56ad1-a56c-4bf6-a05e-bc80eed5b2f8 None admin] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:02.696182 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da56ad1-a56c-4bf6-a05e-bc80eed5b2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:02.696458 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da56ad1-a56c-4bf6-a05e-bc80eed5b2f8 None admin] PATH_INFO: `/v3/groups` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:02.697082 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da56ad1-a56c-4bf6-a05e-bc80eed5b2f8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:02.698521 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da56ad1-a56c-4bf6-a05e-bc80eed5b2f8 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:02.726835 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 164/330] 10.222.0.33 () {60 vars in 1134 bytes} [Sun Sep 11 14:54:02 2022] POST /identity/v3/groups => generated 226 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:54:02.726835 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da56ad1-a56c-4bf6-a05e-bc80eed5b2f8 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:02.767044 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:02.774829 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e51f302-1f93-43ee-b445-59f9a9ca2a5d None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:02.812567 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc88873b-fc7a-4179-a7df-8dd8cab4fcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:02.813288 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88873b-fc7a-4179-a7df-8dd8cab4fcea None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:02.813572 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88873b-fc7a-4179-a7df-8dd8cab4fcea None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:02.813846 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88873b-fc7a-4179-a7df-8dd8cab4fcea None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:02.814534 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc88873b-fc7a-4179-a7df-8dd8cab4fcea None admin] RBAC: Adding query filter params (name=member) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:02.814864 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc88873b-fc7a-4179-a7df-8dd8cab4fcea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:02.816285 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc88873b-fc7a-4179-a7df-8dd8cab4fcea None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:02.825735 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 167/331] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 14:54:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:02.832032 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc88873b-fc7a-4179-a7df-8dd8cab4fcea None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:02.844942 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da56ad1-a56c-4bf6-a05e-bc80eed5b2f8 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:02.883465 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4592d3cd-baab-4dc8-8044-7c5d678bf0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:02.884124 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592d3cd-baab-4dc8-8044-7c5d678bf0f1 None admin] REQUEST_METHOD: `PUT` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:02.884404 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592d3cd-baab-4dc8-8044-7c5d678bf0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:02.884683 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592d3cd-baab-4dc8-8044-7c5d678bf0f1 None admin] PATH_INFO: `/v3/system/users/4f1b2b9a76494f4789a10a53fc478b44/roles/06e98804662441c88b8871c32b9468aa` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:02.886189 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4592d3cd-baab-4dc8-8044-7c5d678bf0f1 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4f1b2b9a76494f4789a10a53fc478b44, role_id=06e98804662441c88b8871c32b9468aa)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:02.887583 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4592d3cd-baab-4dc8-8044-7c5d678bf0f1 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:02.889873 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:02.899907 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 165/332] 10.222.0.33 () {58 vars in 1334 bytes} [Sun Sep 11 14:54:02 2022] PUT /identity/v3/system/users/4f1b2b9a76494f4789a10a53fc478b44/roles/06e98804662441c88b8871c32b9468aa => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:54:02.952653 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc88873b-fc7a-4179-a7df-8dd8cab4fcea None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:03.000763 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc6eb6b-30d1-4193-b566-96c35bc5b6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:03.001421 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6eb6b-30d1-4193-b566-96c35bc5b6ff None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:03.001706 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6eb6b-30d1-4193-b566-96c35bc5b6ff None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:03.001979 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6eb6b-30d1-4193-b566-96c35bc5b6ff None admin] PATH_INFO: `/v3/users/0939a77927c342c4a27d597415b7bf96` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:03.003293 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc6eb6b-30d1-4193-b566-96c35bc5b6ff None admin] RBAC: Authorizing `identity:get_user(user_id=0939a77927c342c4a27d597415b7bf96)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:03.004810 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc6eb6b-30d1-4193-b566-96c35bc5b6ff None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:03.006791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 168/333] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:54:02 2022] GET /identity/v3/users/0939a77927c342c4a27d597415b7bf96 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:54:03.012758 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4592d3cd-baab-4dc8-8044-7c5d678bf0f1 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:03.130306 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4592d3cd-baab-4dc8-8044-7c5d678bf0f1 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:03.166654 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56941c4c-23d2-4f28-9ef6-4ca726a859bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:03.167315 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56941c4c-23d2-4f28-9ef6-4ca726a859bb None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:03.167599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56941c4c-23d2-4f28-9ef6-4ca726a859bb None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:03.167866 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56941c4c-23d2-4f28-9ef6-4ca726a859bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:03.168591 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56941c4c-23d2-4f28-9ef6-4ca726a859bb None admin] RBAC: Adding query filter params (role.id=c9da950c75da4939b8bd649c11015650, user.id=0939a77927c342c4a27d597415b7bf96, scope.system=all) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:03.168898 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56941c4c-23d2-4f28-9ef6-4ca726a859bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:03.170357 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56941c4c-23d2-4f28-9ef6-4ca726a859bb None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:03.183420 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 166/334] 10.222.0.33 () {56 vars in 1308 bytes} [Sun Sep 11 14:54:03 2022] GET /identity/v3/role_assignments?user.id=0939a77927c342c4a27d597415b7bf96&scope.system=all&role.id=c9da950c75da4939b8bd649c11015650 => generated 510 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:05.399115 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a501a3-fdbd-41ce-bdfd-6222ebf5ccf1 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:05.399634 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a501a3-fdbd-41ce-bdfd-6222ebf5ccf1 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:05.399634 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a501a3-fdbd-41ce-bdfd-6222ebf5ccf1 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:05.401294 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 169/335] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:05.408949 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5927cbf-3d62-488f-86fd-532a4a3a9b80 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:05.409442 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5927cbf-3d62-488f-86fd-532a4a3a9b80 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:05.409442 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5927cbf-3d62-488f-86fd-532a4a3a9b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:05.435074 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-e5927cbf-3d62-488f-86fd-532a4a3a9b80 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:05.436789 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-e5927cbf-3d62-488f-86fd-532a4a3a9b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:05.509603 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:05.530583 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 167/336] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:54:05.539850 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbea66-aee6-4a8f-abf7-b5dcb820dde5 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:05.540136 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbea66-aee6-4a8f-abf7-b5dcb820dde5 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:05.540414 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbea66-aee6-4a8f-abf7-b5dcb820dde5 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:05.542130 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 170/337] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:05.548637 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ed8a1-fd63-4b8e-a069-2a8f88c2a14a None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:05.548989 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ed8a1-fd63-4b8e-a069-2a8f88c2a14a None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:05.549315 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ed8a1-fd63-4b8e-a069-2a8f88c2a14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:05.578850 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-f58ed8a1-fd63-4b8e-a069-2a8f88c2a14a None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:05.580092 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-f58ed8a1-fd63-4b8e-a069-2a8f88c2a14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:05.671321 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 168/338] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:05.920475 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebbea66-aee6-4a8f-abf7-b5dcb820dde5 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:05.949300 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:06.044152 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebbea66-aee6-4a8f-abf7-b5dcb820dde5 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:06.085423 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21bff7d-d938-49bc-97bf-b5853de8945b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:06.086059 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21bff7d-d938-49bc-97bf-b5853de8945b None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:06.086343 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21bff7d-d938-49bc-97bf-b5853de8945b None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:06.086618 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21bff7d-d938-49bc-97bf-b5853de8945b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:06.105132 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21bff7d-d938-49bc-97bf-b5853de8945b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:06.110676 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-a21bff7d-d938-49bc-97bf-b5853de8945b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:54:06.110676 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:06.110676 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:06.110676 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:06.110676 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:06.110676 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:06.110676 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:06.110676 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:06.110676 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:06.110676 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:06.110676 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:06.110676 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:06.110676 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:06.110676 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:06.110676 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:06.110676 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:06.111607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:06.111607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:06.111607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:06.111607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:06.111607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:06.111607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:06.111607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:06.111607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:06.111607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:06.111607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:06.111607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:06.111607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:06.111607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:06.111607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:06.111607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:06.111607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:06.112414 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:06.112414 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:06.112414 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:06.112414 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:06.112414 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:06.112414 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:06.112414 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:06.112414 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:06.112414 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:06.112414 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:06.112414 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:54:06.112414 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:06.113029 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 171/339] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:54:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:06.117561 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ed8a1-fd63-4b8e-a069-2a8f88c2a14a None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:06.237668 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ed8a1-fd63-4b8e-a069-2a8f88c2a14a None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:06.242878 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21bff7d-d938-49bc-97bf-b5853de8945b None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:06.273389 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:06.325773 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50727c6b-8943-414b-bed6-9e1be726feb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:06.326403 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50727c6b-8943-414b-bed6-9e1be726feb1 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:06.326682 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50727c6b-8943-414b-bed6-9e1be726feb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:06.327019 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50727c6b-8943-414b-bed6-9e1be726feb1 None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:06.327666 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50727c6b-8943-414b-bed6-9e1be726feb1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:06.328149 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50727c6b-8943-414b-bed6-9e1be726feb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:06.329604 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50727c6b-8943-414b-bed6-9e1be726feb1 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:06.345989 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 169/340] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 14:54:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:06.351231 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50727c6b-8943-414b-bed6-9e1be726feb1 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:06.417603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:06.438428 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21bff7d-d938-49bc-97bf-b5853de8945b None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:06.496741 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63314dd6-8aee-44c8-bb2e-6427ed5579c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:06.497321 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63314dd6-8aee-44c8-bb2e-6427ed5579c0 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:06.497605 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63314dd6-8aee-44c8-bb2e-6427ed5579c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:06.498029 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63314dd6-8aee-44c8-bb2e-6427ed5579c0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:06.508399 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50727c6b-8943-414b-bed6-9e1be726feb1 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:06.516097 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63314dd6-8aee-44c8-bb2e-6427ed5579c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:06.522113 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-63314dd6-8aee-44c8-bb2e-6427ed5579c0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:54:06.522113 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:06.522113 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:06.522113 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:06.522113 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:06.522113 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:06.522113 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:06.522113 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:06.522113 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:06.522113 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:06.522113 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:06.522113 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:06.522113 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:06.522113 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:06.522113 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:06.522113 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:06.523087 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:06.523087 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:06.523087 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:06.523087 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:06.523087 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:06.523087 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:06.523087 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:06.523087 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:06.523087 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:06.523087 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:06.523087 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:06.523087 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:06.523087 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:06.523087 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:06.523087 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:06.523087 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:06.524611 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:06.524611 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:06.524611 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:06.524611 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:06.524611 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:06.524611 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:06.524611 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:06.524611 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:06.524611 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:06.524611 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:06.524611 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:54:06.524611 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:06.524611 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 172/341] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:54:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:06.525422 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f7e6f-6a5c-46c6-bd0d-d72b51843c0e None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:06.525588 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f7e6f-6a5c-46c6-bd0d-d72b51843c0e None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:06.526681 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f7e6f-6a5c-46c6-bd0d-d72b51843c0e None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:06.527755 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 173/342] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:06.529262 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f4aa0-4158-4e2a-9312-27b1eb595f4d None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:06.529555 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f4aa0-4158-4e2a-9312-27b1eb595f4d None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:06.529833 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f4aa0-4158-4e2a-9312-27b1eb595f4d None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:06.532420 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 174/343] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:06.533930 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a83d7-1389-41cc-a2de-e4a1397a57ef None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:06.534222 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a83d7-1389-41cc-a2de-e4a1397a57ef None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:06.534496 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a83d7-1389-41cc-a2de-e4a1397a57ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:06.553636 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c624f8-d24c-446c-acaa-792c67032ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:06.554438 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c624f8-d24c-446c-acaa-792c67032ae7 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:06.554787 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c624f8-d24c-446c-acaa-792c67032ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:06.555168 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c624f8-d24c-446c-acaa-792c67032ae7 None admin] PATH_INFO: `/v3/users/4f1b2b9a76494f4789a10a53fc478b44` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:06.556434 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c624f8-d24c-446c-acaa-792c67032ae7 None admin] RBAC: Authorizing `identity:get_user(user_id=4f1b2b9a76494f4789a10a53fc478b44)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:06.558229 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c624f8-d24c-446c-acaa-792c67032ae7 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:06.558430 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-ce1a83d7-1389-41cc-a2de-e4a1397a57ef None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:06.559715 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-ce1a83d7-1389-41cc-a2de-e4a1397a57ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:06.560089 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 170/344] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:54:06 2022] GET /identity/v3/users/4f1b2b9a76494f4789a10a53fc478b44 => generated 293 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:54:06.561288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2853b-f46f-484b-96ff-a75cc2ca3536 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:06.561574 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2853b-f46f-484b-96ff-a75cc2ca3536 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:06.562159 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2853b-f46f-484b-96ff-a75cc2ca3536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:06.586798 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-a6e2853b-f46f-484b-96ff-a75cc2ca3536 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:06.588621 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-a6e2853b-f46f-484b-96ff-a75cc2ca3536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:06.645699 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:06.653154 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 175/345] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:54:06.654255 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a83d7-1389-41cc-a2de-e4a1397a57ef None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:06.688874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 171/346] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:06.689861 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e2853b-f46f-484b-96ff-a75cc2ca3536 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:06.820540 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:06.864134 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a83d7-1389-41cc-a2de-e4a1397a57ef None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:06.925851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45648713-d2eb-46e9-bddc-cf6d71e6b08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:06.926476 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45648713-d2eb-46e9-bddc-cf6d71e6b08c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:06.926758 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45648713-d2eb-46e9-bddc-cf6d71e6b08c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:06.927063 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45648713-d2eb-46e9-bddc-cf6d71e6b08c None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:06.927737 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45648713-d2eb-46e9-bddc-cf6d71e6b08c None admin] RBAC: Adding query filter params (name=member) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:06.928043 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45648713-d2eb-46e9-bddc-cf6d71e6b08c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:06.931978 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45648713-d2eb-46e9-bddc-cf6d71e6b08c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:06.950322 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 176/347] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 14:54:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:06.954219 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e2853b-f46f-484b-96ff-a75cc2ca3536 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:06.966564 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45648713-d2eb-46e9-bddc-cf6d71e6b08c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:07.044591 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85a66b-a9a6-4f9f-b51f-8117924f19c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:07.045216 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85a66b-a9a6-4f9f-b51f-8117924f19c3 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:07.045494 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85a66b-a9a6-4f9f-b51f-8117924f19c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:07.045764 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85a66b-a9a6-4f9f-b51f-8117924f19c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:07.046496 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b85a66b-a9a6-4f9f-b51f-8117924f19c3 None admin] RBAC: Adding query filter params (role.id=06e98804662441c88b8871c32b9468aa, user.id=4f1b2b9a76494f4789a10a53fc478b44, scope.system=all) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:07.046881 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b85a66b-a9a6-4f9f-b51f-8117924f19c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:07.048321 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b85a66b-a9a6-4f9f-b51f-8117924f19c3 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:07.067171 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 172/348] 10.222.0.33 () {56 vars in 1308 bytes} [Sun Sep 11 14:54:06 2022] GET /identity/v3/role_assignments?user.id=4f1b2b9a76494f4789a10a53fc478b44&scope.system=all&role.id=06e98804662441c88b8871c32b9468aa => generated 510 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:07.083897 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85a66b-a9a6-4f9f-b51f-8117924f19c3 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:07.144316 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:07.155371 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45648713-d2eb-46e9-bddc-cf6d71e6b08c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:07.202215 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d273f-690c-43ed-8ffc-5f66ce925318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:07.204138 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d273f-690c-43ed-8ffc-5f66ce925318 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:07.204424 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d273f-690c-43ed-8ffc-5f66ce925318 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:07.204699 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d273f-690c-43ed-8ffc-5f66ce925318 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:07.215270 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:07.262054 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85a66b-a9a6-4f9f-b51f-8117924f19c3 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:07.304737 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a3b1d-f806-4591-846b-bc1f5f2c5e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:07.305917 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a3b1d-f806-4591-846b-bc1f5f2c5e8b None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:07.306450 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a3b1d-f806-4591-846b-bc1f5f2c5e8b None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:07.306987 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a3b1d-f806-4591-846b-bc1f5f2c5e8b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:07.323291 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a3b1d-f806-4591-846b-bc1f5f2c5e8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:07.330307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-496a3b1d-f806-4591-846b-bc1f5f2c5e8b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:54:07.330307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:07.330307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:07.330307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:07.330307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:07.330307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:07.330307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:07.330307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:07.330307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:07.330307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:07.330307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:07.330307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:07.330307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:07.330307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:07.330307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:07.330307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:07.331385 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:07.331385 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:07.331385 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:07.331385 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:07.331385 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:07.331385 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:07.331385 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:07.331385 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:07.331385 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:07.331385 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:07.331385 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:07.331385 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:07.331385 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:07.331385 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:07.331385 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:07.331385 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:07.332256 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:07.332256 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:07.332256 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:07.332256 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:07.332256 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:07.332256 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:07.332256 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:07.332256 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:07.332256 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:07.332256 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:07.332256 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:54:07.332256 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:07.333302 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 173/349] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 14:54:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:07.334006 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a3b1d-f806-4591-846b-bc1f5f2c5e8b None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:07.342521 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d273f-690c-43ed-8ffc-5f66ce925318 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:07.347376 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d273f-690c-43ed-8ffc-5f66ce925318 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:07.476184 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:07.486679 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a3b1d-f806-4591-846b-bc1f5f2c5e8b None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:07.490658 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 177/350] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:54:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:54:07.492730 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d273f-690c-43ed-8ffc-5f66ce925318 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:07.541291 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbfa268-9356-470b-8dd6-bdf5adf79b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:07.541991 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbfa268-9356-470b-8dd6-bdf5adf79b96 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:07.542210 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbfa268-9356-470b-8dd6-bdf5adf79b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:07.542477 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbfa268-9356-470b-8dd6-bdf5adf79b96 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:07.546374 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:07.560338 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbfa268-9356-470b-8dd6-bdf5adf79b96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:07.566408 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-5dbfa268-9356-470b-8dd6-bdf5adf79b96 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:54:07.566408 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:07.566408 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:07.566408 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:07.566408 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:07.566408 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:07.566408 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:07.566408 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:07.566408 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:07.566408 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:07.566408 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:07.566408 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:07.566408 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:07.566408 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:07.566408 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:07.566408 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:07.567390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:07.567390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:07.567390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:07.567390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:07.567390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:07.567390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:07.567390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:07.567390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:07.567390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:07.567390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:07.567390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:07.567390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:07.567390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:07.567390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:07.567390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:07.567390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:07.568207 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:07.568207 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:07.568207 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:07.568207 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:07.568207 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:07.568207 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:07.568207 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:07.568207 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:07.568207 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:07.568207 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:07.568207 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:54:07.568207 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:07.568207 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 174/351] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:54:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:07.568904 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbfa268-9356-470b-8dd6-bdf5adf79b96 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:07.649783 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d273f-690c-43ed-8ffc-5f66ce925318 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:07.694916 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4250dba-dfcf-41b5-a072-bee603ac240b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:07.695535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4250dba-dfcf-41b5-a072-bee603ac240b None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:07.695811 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4250dba-dfcf-41b5-a072-bee603ac240b None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:07.696080 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4250dba-dfcf-41b5-a072-bee603ac240b None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:07.696804 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4250dba-dfcf-41b5-a072-bee603ac240b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:07.697253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4250dba-dfcf-41b5-a072-bee603ac240b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:07.698622 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4250dba-dfcf-41b5-a072-bee603ac240b None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:07.708140 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 178/352] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:54:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:07.708981 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4250dba-dfcf-41b5-a072-bee603ac240b None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:07.709459 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbfa268-9356-470b-8dd6-bdf5adf79b96 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:07.760265 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540e9a6-1f52-4792-a4e7-81a724eb024c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:07.760938 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540e9a6-1f52-4792-a4e7-81a724eb024c None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:07.761222 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540e9a6-1f52-4792-a4e7-81a724eb024c None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:07.761497 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540e9a6-1f52-4792-a4e7-81a724eb024c None admin] PATH_INFO: `/v3/projects/3b402af2708b41df849963f81e65797e` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:07.763548 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e540e9a6-1f52-4792-a4e7-81a724eb024c None admin] RBAC: Authorizing `identity:get_project(project_id=3b402af2708b41df849963f81e65797e)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:07.765388 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e540e9a6-1f52-4792-a4e7-81a724eb024c None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:07.768399 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 175/353] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:54:07 2022] GET /identity/v3/projects/3b402af2708b41df849963f81e65797e => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:07.769473 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540e9a6-1f52-4792-a4e7-81a724eb024c None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:07.796252 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:07.831349 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:07.838641 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4250dba-dfcf-41b5-a072-bee603ac240b None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:07.878268 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da7d39-25dc-4d39-8fda-24596bb186ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:07.881463 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da7d39-25dc-4d39-8fda-24596bb186ae None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:07.881463 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da7d39-25dc-4d39-8fda-24596bb186ae None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:07.881463 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da7d39-25dc-4d39-8fda-24596bb186ae None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:07.881463 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59da7d39-25dc-4d39-8fda-24596bb186ae None admin] RBAC: Adding query filter params (name=member) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:07.881463 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59da7d39-25dc-4d39-8fda-24596bb186ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:07.881855 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59da7d39-25dc-4d39-8fda-24596bb186ae None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:07.891169 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 179/354] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 14:54:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:07.892237 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540e9a6-1f52-4792-a4e7-81a724eb024c None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:07.892548 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d7c87-ef02-45ad-84ab-f1aeecc9c294 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:07.892844 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d7c87-ef02-45ad-84ab-f1aeecc9c294 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:07.893124 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d7c87-ef02-45ad-84ab-f1aeecc9c294 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:07.894487 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 180/355] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:07.899246 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d7c87-ef02-45ad-84ab-f1aeecc9c294 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:07.932370 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f73a835-4273-4191-9541-ba510b844687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:07.932370 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f73a835-4273-4191-9541-ba510b844687 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:07.932634 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f73a835-4273-4191-9541-ba510b844687 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:07.932968 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f73a835-4273-4191-9541-ba510b844687 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:08.023623 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f73a835-4273-4191-9541-ba510b844687 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:08.025127 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f73a835-4273-4191-9541-ba510b844687 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:08.035006 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d7c87-ef02-45ad-84ab-f1aeecc9c294 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:08.055271 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:08.074461 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070caa4-61f2-4d70-a487-524b6e06ea50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:08.075136 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070caa4-61f2-4d70-a487-524b6e06ea50 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:08.075938 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070caa4-61f2-4d70-a487-524b6e06ea50 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:08.075938 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070caa4-61f2-4d70-a487-524b6e06ea50 None admin] PATH_INFO: `/v3/groups/937e7e73e3f3416a83b83b64423e5bea` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:08.084079 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4070caa4-61f2-4d70-a487-524b6e06ea50 None admin] RBAC: Authorizing `identity:get_group(group_id=937e7e73e3f3416a83b83b64423e5bea)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:08.085610 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4070caa4-61f2-4d70-a487-524b6e06ea50 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:08.087311 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 181/356] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 11 14:54:07 2022] GET /identity/v3/groups/937e7e73e3f3416a83b83b64423e5bea => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:08.088485 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa76d4-9f2d-4f1d-8e20-d9b5e3e39780 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:08.088774 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa76d4-9f2d-4f1d-8e20-d9b5e3e39780 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:08.089053 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa76d4-9f2d-4f1d-8e20-d9b5e3e39780 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:08.090267 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 182/357] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:08.091999 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa76d4-9f2d-4f1d-8e20-d9b5e3e39780 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:08.125841 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:08.158996 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 176/358] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:54:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:54:08.160535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e8560-c7a8-4eb2-9cdd-f7c8808dc560 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:08.161009 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e8560-c7a8-4eb2-9cdd-f7c8808dc560 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:08.161428 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e8560-c7a8-4eb2-9cdd-f7c8808dc560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:08.181773 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-047e8560-c7a8-4eb2-9cdd-f7c8808dc560 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:08.183363 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-047e8560-c7a8-4eb2-9cdd-f7c8808dc560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:08.220533 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa76d4-9f2d-4f1d-8e20-d9b5e3e39780 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:08.259123 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff687f6e-3587-4282-8adb-7ec84854c0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:08.259802 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff687f6e-3587-4282-8adb-7ec84854c0c5 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:08.260087 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff687f6e-3587-4282-8adb-7ec84854c0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:08.260363 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff687f6e-3587-4282-8adb-7ec84854c0c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:08.270041 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 177/359] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:08.270923 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047e8560-c7a8-4eb2-9cdd-f7c8808dc560 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:08.346455 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:08.347671 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff687f6e-3587-4282-8adb-7ec84854c0c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:08.349458 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff687f6e-3587-4282-8adb-7ec84854c0c5 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:08.380230 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:08.408838 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047e8560-c7a8-4eb2-9cdd-f7c8808dc560 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:08.447688 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7aa295-dee1-4441-9c0f-0aee49df7125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:08.448466 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7aa295-dee1-4441-9c0f-0aee49df7125 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:08.448584 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7aa295-dee1-4441-9c0f-0aee49df7125 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:08.448797 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7aa295-dee1-4441-9c0f-0aee49df7125 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:08.449598 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7aa295-dee1-4441-9c0f-0aee49df7125 None admin] RBAC: Adding query filter params (group.id=937e7e73e3f3416a83b83b64423e5bea, role.id=c9da950c75da4939b8bd649c11015650, scope.project.id=3b402af2708b41df849963f81e65797e) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:08.449904 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7aa295-dee1-4441-9c0f-0aee49df7125 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:08.451403 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7aa295-dee1-4441-9c0f-0aee49df7125 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:08.457803 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 178/360] 10.222.0.33 () {56 vars in 1376 bytes} [Sun Sep 11 14:54:08 2022] GET /identity/v3/role_assignments?group.id=937e7e73e3f3416a83b83b64423e5bea&scope.project.id=3b402af2708b41df849963f81e65797e&role.id=c9da950c75da4939b8bd649c11015650 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:08.459123 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1b264-0e25-4400-a8d3-c8240f5dbe1c None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:08.459413 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1b264-0e25-4400-a8d3-c8240f5dbe1c None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:08.459688 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1b264-0e25-4400-a8d3-c8240f5dbe1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:08.483299 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 183/361] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:54:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:54:08.485180 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff687f6e-3587-4282-8adb-7ec84854c0c5 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:08.502036 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-c1a1b264-0e25-4400-a8d3-c8240f5dbe1c None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:08.503310 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-c1a1b264-0e25-4400-a8d3-c8240f5dbe1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:08.611656 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 179/362] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:08.613578 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1b264-0e25-4400-a8d3-c8240f5dbe1c None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:08.628507 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff687f6e-3587-4282-8adb-7ec84854c0c5 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:08.667572 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:08.681153 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:08.683391 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebfe10-7026-467b-afc2-3fd8b486bc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:08.684026 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebfe10-7026-467b-afc2-3fd8b486bc61 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:08.684438 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebfe10-7026-467b-afc2-3fd8b486bc61 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:08.684650 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebfe10-7026-467b-afc2-3fd8b486bc61 None admin] PATH_INFO: `/v3/projects/3b402af2708b41df849963f81e65797e` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:08.685728 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bebfe10-7026-467b-afc2-3fd8b486bc61 None admin] RBAC: Authorizing `identity:get_project(project_id=3b402af2708b41df849963f81e65797e)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:08.688287 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bebfe10-7026-467b-afc2-3fd8b486bc61 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:08.689822 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 184/363] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:54:08 2022] GET /identity/v3/projects/3b402af2708b41df849963f81e65797e => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:08.690597 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebfe10-7026-467b-afc2-3fd8b486bc61 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:08.747586 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1b264-0e25-4400-a8d3-c8240f5dbe1c None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:08.795993 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfef53-4981-4c3e-8662-747497b373e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:08.796770 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfef53-4981-4c3e-8662-747497b373e3 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:08.797052 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfef53-4981-4c3e-8662-747497b373e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:08.797325 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfef53-4981-4c3e-8662-747497b373e3 None admin] PATH_INFO: `/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:08.798580 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cfef53-4981-4c3e-8662-747497b373e3 None admin] RBAC: Authorizing `identity:get_project(project_id=8f8cdaafdd52411bb20efe51ddc2565c)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:08.800227 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cfef53-4981-4c3e-8662-747497b373e3 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:08.802191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 180/364] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:54:08 2022] GET /identity/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:08.803014 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfef53-4981-4c3e-8662-747497b373e3 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:08.821920 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebfe10-7026-467b-afc2-3fd8b486bc61 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:08.871333 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01846c-b724-45e5-84f4-bf6e0981decd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:08.872828 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01846c-b724-45e5-84f4-bf6e0981decd None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:08.873434 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01846c-b724-45e5-84f4-bf6e0981decd None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:08.873977 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01846c-b724-45e5-84f4-bf6e0981decd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:08.889886 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d01846c-b724-45e5-84f4-bf6e0981decd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:08.897088 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-2d01846c-b724-45e5-84f4-bf6e0981decd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:54:08.897088 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:08.897088 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:08.897088 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:08.897088 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:08.897088 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:08.897088 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:08.897088 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:08.897088 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:08.897088 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:08.897088 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:08.897088 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:08.897088 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:08.897088 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:08.897088 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:08.897088 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:08.898036 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:08.898036 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:08.898036 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:08.898036 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:08.898036 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:08.898036 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:08.898036 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:08.898036 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:08.898036 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:08.898036 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:08.898036 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:08.898036 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:08.898036 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:08.898036 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:08.898036 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:08.898036 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:08.898870 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:08.898870 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:08.898870 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:08.898870 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:08.898870 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:08.898870 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:08.898870 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:08.898870 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:08.898870 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:08.898870 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:08.898870 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:54:08.898870 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:08.900370 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 185/365] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 14:54:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:08.901184 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01846c-b724-45e5-84f4-bf6e0981decd None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:08.939355 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfef53-4981-4c3e-8662-747497b373e3 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:08.956265 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:08.968235 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:08.988408 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e4c782-dcb3-470c-9f5a-eb6205ce5180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:08.989199 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4c782-dcb3-470c-9f5a-eb6205ce5180 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:08.989566 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4c782-dcb3-470c-9f5a-eb6205ce5180 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:08.989879 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4c782-dcb3-470c-9f5a-eb6205ce5180 None admin] PATH_INFO: `/v3/groups/937e7e73e3f3416a83b83b64423e5bea` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:08.991080 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e4c782-dcb3-470c-9f5a-eb6205ce5180 None admin] RBAC: Authorizing `identity:get_group(group_id=937e7e73e3f3416a83b83b64423e5bea)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:08.993241 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e4c782-dcb3-470c-9f5a-eb6205ce5180 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:08.999343 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 181/366] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 11 14:54:08 2022] GET /identity/v3/groups/937e7e73e3f3416a83b83b64423e5bea => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:08.999343 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e4c782-dcb3-470c-9f5a-eb6205ce5180 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:09.041021 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01846c-b724-45e5-84f4-bf6e0981decd None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:09.084880 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681800b8-0eaa-4065-836e-3c9677f3cb78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:09.086099 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681800b8-0eaa-4065-836e-3c9677f3cb78 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:09.086390 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681800b8-0eaa-4065-836e-3c9677f3cb78 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:09.086663 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681800b8-0eaa-4065-836e-3c9677f3cb78 None admin] PATH_INFO: `/v3/groups/937e7e73e3f3416a83b83b64423e5bea` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:09.088149 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681800b8-0eaa-4065-836e-3c9677f3cb78 None admin] RBAC: Authorizing `identity:get_group(group_id=937e7e73e3f3416a83b83b64423e5bea)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:09.090625 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681800b8-0eaa-4065-836e-3c9677f3cb78 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:09.092321 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 186/367] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 11 14:54:08 2022] GET /identity/v3/groups/937e7e73e3f3416a83b83b64423e5bea => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:09.093092 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681800b8-0eaa-4065-836e-3c9677f3cb78 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:09.130404 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e4c782-dcb3-470c-9f5a-eb6205ce5180 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:09.170686 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e77009-6019-42d0-9921-4440ed7fd153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:09.171436 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e77009-6019-42d0-9921-4440ed7fd153 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:09.172255 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e77009-6019-42d0-9921-4440ed7fd153 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:09.172606 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e77009-6019-42d0-9921-4440ed7fd153 None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:09.173376 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e77009-6019-42d0-9921-4440ed7fd153 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:09.173752 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e77009-6019-42d0-9921-4440ed7fd153 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:09.175362 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e77009-6019-42d0-9921-4440ed7fd153 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:09.184744 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 182/368] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:54:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:09.185348 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e77009-6019-42d0-9921-4440ed7fd153 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:09.221071 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681800b8-0eaa-4065-836e-3c9677f3cb78 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:09.244778 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:09.248392 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:09.272250 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad9054-78b2-4b15-9653-721badf0d9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:09.272813 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad9054-78b2-4b15-9653-721badf0d9d5 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:09.273133 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad9054-78b2-4b15-9653-721badf0d9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:09.273359 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad9054-78b2-4b15-9653-721badf0d9d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:09.289654 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ad9054-78b2-4b15-9653-721badf0d9d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:09.294670 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-c3ad9054-78b2-4b15-9653-721badf0d9d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:54:09.294670 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:09.294670 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:09.294670 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:09.294670 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:09.294670 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:09.294670 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:09.294670 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:09.294670 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:09.294670 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:09.294670 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:09.294670 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:09.294670 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:09.294670 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:09.294670 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:09.294670 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:09.295650 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:09.295650 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:09.295650 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:09.295650 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:09.295650 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:09.295650 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:09.295650 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:09.295650 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:09.295650 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:09.295650 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:09.295650 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:09.295650 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:09.295650 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:09.295650 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:09.295650 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:09.295650 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:09.302633 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:09.302633 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:09.302633 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:09.302633 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:09.302633 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:09.302633 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:09.302633 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:09.302633 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:09.302633 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:09.302633 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:09.302633 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:54:09.302633 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:09.302633 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 187/369] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:54:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:09.302633 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad9054-78b2-4b15-9653-721badf0d9d5 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:09.319776 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e77009-6019-42d0-9921-4440ed7fd153 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:09.365062 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87031ac-75ee-49a3-aaca-990396078fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:09.365601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87031ac-75ee-49a3-aaca-990396078fd1 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:09.365885 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87031ac-75ee-49a3-aaca-990396078fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:09.366161 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87031ac-75ee-49a3-aaca-990396078fd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:09.366937 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87031ac-75ee-49a3-aaca-990396078fd1 None admin] RBAC: Adding query filter params (group.id=937e7e73e3f3416a83b83b64423e5bea, role.id=d1818eb7f2bd467fbf75bad5cc8e0aea, scope.project.id=3b402af2708b41df849963f81e65797e) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:09.367254 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87031ac-75ee-49a3-aaca-990396078fd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:09.368970 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87031ac-75ee-49a3-aaca-990396078fd1 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:09.376358 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 183/370] 10.222.0.33 () {56 vars in 1376 bytes} [Sun Sep 11 14:54:09 2022] GET /identity/v3/role_assignments?group.id=937e7e73e3f3416a83b83b64423e5bea&scope.project.id=3b402af2708b41df849963f81e65797e&role.id=d1818eb7f2bd467fbf75bad5cc8e0aea => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:09.377288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87031ac-75ee-49a3-aaca-990396078fd1 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:09.428167 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad9054-78b2-4b15-9653-721badf0d9d5 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:09.486380 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df0f4a5-fdc8-48ca-9c8e-4d8daceb77a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:09.487443 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0f4a5-fdc8-48ca-9c8e-4d8daceb77a6 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:09.487987 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0f4a5-fdc8-48ca-9c8e-4d8daceb77a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:09.488659 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0f4a5-fdc8-48ca-9c8e-4d8daceb77a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:09.489624 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df0f4a5-fdc8-48ca-9c8e-4d8daceb77a6 None admin] RBAC: Adding query filter params (group.id=937e7e73e3f3416a83b83b64423e5bea, role.id=c9da950c75da4939b8bd649c11015650, scope.project.id=8f8cdaafdd52411bb20efe51ddc2565c) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:09.490144 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df0f4a5-fdc8-48ca-9c8e-4d8daceb77a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:09.491971 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df0f4a5-fdc8-48ca-9c8e-4d8daceb77a6 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:09.500927 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 188/371] 10.222.0.33 () {56 vars in 1376 bytes} [Sun Sep 11 14:54:09 2022] GET /identity/v3/role_assignments?group.id=937e7e73e3f3416a83b83b64423e5bea&scope.project.id=8f8cdaafdd52411bb20efe51ddc2565c&role.id=c9da950c75da4939b8bd649c11015650 => generated 260 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:09.502208 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df0f4a5-fdc8-48ca-9c8e-4d8daceb77a6 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:09.525080 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87031ac-75ee-49a3-aaca-990396078fd1 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:09.544107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:09.594451 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:09.600589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f8037-fb35-4282-ae77-0359d7e7e893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:09.601475 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f8037-fb35-4282-ae77-0359d7e7e893 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:09.601977 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f8037-fb35-4282-ae77-0359d7e7e893 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:09.602450 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f8037-fb35-4282-ae77-0359d7e7e893 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:09.660933 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df0f4a5-fdc8-48ca-9c8e-4d8daceb77a6 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:09.685548 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f8037-fb35-4282-ae77-0359d7e7e893 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:09.688771 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f8037-fb35-4282-ae77-0359d7e7e893 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:09.702492 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42fcc2f-5755-4041-aaa2-ba9fd7d405bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:09.703167 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fcc2f-5755-4041-aaa2-ba9fd7d405bc None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:09.703435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fcc2f-5755-4041-aaa2-ba9fd7d405bc None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:09.703701 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fcc2f-5755-4041-aaa2-ba9fd7d405bc None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:09.704386 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42fcc2f-5755-4041-aaa2-ba9fd7d405bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:09.704691 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42fcc2f-5755-4041-aaa2-ba9fd7d405bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:09.706117 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42fcc2f-5755-4041-aaa2-ba9fd7d405bc None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:09.716025 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 189/372] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:54:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:09.722373 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42fcc2f-5755-4041-aaa2-ba9fd7d405bc None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:09.803432 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 184/373] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:54:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:54:09.812954 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f8037-fb35-4282-ae77-0359d7e7e893 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:09.823546 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:09.845334 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42fcc2f-5755-4041-aaa2-ba9fd7d405bc None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:09.853026 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:09.891608 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a46ef-8901-4419-9e4a-20948f90c40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:09.892242 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a46ef-8901-4419-9e4a-20948f90c40f None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:09.892529 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a46ef-8901-4419-9e4a-20948f90c40f None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:09.893239 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a46ef-8901-4419-9e4a-20948f90c40f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:09.941233 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f8037-fb35-4282-ae77-0359d7e7e893 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:09.974855 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a46ef-8901-4419-9e4a-20948f90c40f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:09.976483 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a46ef-8901-4419-9e4a-20948f90c40f None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:09.994639 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fff98c-cd03-41b2-b4f7-27d845673c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:09.995605 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fff98c-cd03-41b2-b4f7-27d845673c37 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:09.996079 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fff98c-cd03-41b2-b4f7-27d845673c37 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:09.996587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fff98c-cd03-41b2-b4f7-27d845673c37 None admin] PATH_INFO: `/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:10.007647 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fff98c-cd03-41b2-b4f7-27d845673c37 None admin] RBAC: Authorizing `identity:get_project(project_id=8f8cdaafdd52411bb20efe51ddc2565c)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:10.009133 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fff98c-cd03-41b2-b4f7-27d845673c37 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:10.010896 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 185/374] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:54:09 2022] GET /identity/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:54:10.031924 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fff98c-cd03-41b2-b4f7-27d845673c37 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:10.091635 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 190/375] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:54:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:54:10.097262 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a46ef-8901-4419-9e4a-20948f90c40f None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:10.105322 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:10.175365 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:10.223286 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fff98c-cd03-41b2-b4f7-27d845673c37 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:10.275899 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a46ef-8901-4419-9e4a-20948f90c40f None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:10.312900 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0d494-2d17-4bc2-a8af-7938d8b5b724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:10.313551 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0d494-2d17-4bc2-a8af-7938d8b5b724 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:10.313834 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0d494-2d17-4bc2-a8af-7938d8b5b724 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:10.314108 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0d494-2d17-4bc2-a8af-7938d8b5b724 None admin] PATH_INFO: `/v3/groups/937e7e73e3f3416a83b83b64423e5bea` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:10.315685 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb0d494-2d17-4bc2-a8af-7938d8b5b724 None admin] RBAC: Authorizing `identity:get_group(group_id=937e7e73e3f3416a83b83b64423e5bea)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:10.317166 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb0d494-2d17-4bc2-a8af-7938d8b5b724 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:10.320658 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e3aed-7018-46d4-b1b1-bf4f8608d600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:10.321303 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e3aed-7018-46d4-b1b1-bf4f8608d600 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:10.321580 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e3aed-7018-46d4-b1b1-bf4f8608d600 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:10.321857 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e3aed-7018-46d4-b1b1-bf4f8608d600 None admin] PATH_INFO: `/v3/projects/b748811b1c8a461188c96c2e314bf507` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:10.322385 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 186/376] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 11 14:54:10 2022] GET /identity/v3/groups/937e7e73e3f3416a83b83b64423e5bea => generated 226 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:10.323026 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e3aed-7018-46d4-b1b1-bf4f8608d600 None admin] RBAC: Authorizing `identity:get_project(project_id=b748811b1c8a461188c96c2e314bf507)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:10.324600 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e3aed-7018-46d4-b1b1-bf4f8608d600 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:10.327138 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0d494-2d17-4bc2-a8af-7938d8b5b724 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:10.331992 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 191/377] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:54:10 2022] GET /identity/v3/projects/b748811b1c8a461188c96c2e314bf507 => generated 344 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:54:10.335332 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e3aed-7018-46d4-b1b1-bf4f8608d600 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:10.423273 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:10.453565 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e3aed-7018-46d4-b1b1-bf4f8608d600 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:10.489522 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0d494-2d17-4bc2-a8af-7938d8b5b724 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:10.507572 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51daf012-55f2-4123-8782-9dfaddcb678d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:10.508479 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51daf012-55f2-4123-8782-9dfaddcb678d None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:10.508934 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51daf012-55f2-4123-8782-9dfaddcb678d None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:10.509356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51daf012-55f2-4123-8782-9dfaddcb678d None admin] PATH_INFO: `/v3/groups/b87501c1d6954becaf4c80c640bad653` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:10.519492 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51daf012-55f2-4123-8782-9dfaddcb678d None admin] RBAC: Authorizing `identity:get_group(group_id=b87501c1d6954becaf4c80c640bad653)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:10.521255 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51daf012-55f2-4123-8782-9dfaddcb678d None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:10.523520 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 192/378] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 11 14:54:10 2022] GET /identity/v3/groups/b87501c1d6954becaf4c80c640bad653 => generated 229 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:10.528883 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51daf012-55f2-4123-8782-9dfaddcb678d None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:10.535776 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681983b2-d3c0-4878-8d33-c2762b0dd755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:10.536451 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681983b2-d3c0-4878-8d33-c2762b0dd755 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:10.536731 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681983b2-d3c0-4878-8d33-c2762b0dd755 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:10.537055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681983b2-d3c0-4878-8d33-c2762b0dd755 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:10.537799 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681983b2-d3c0-4878-8d33-c2762b0dd755 None admin] RBAC: Adding query filter params (group.id=937e7e73e3f3416a83b83b64423e5bea, role.id=d1818eb7f2bd467fbf75bad5cc8e0aea, scope.project.id=8f8cdaafdd52411bb20efe51ddc2565c) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:10.538106 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681983b2-d3c0-4878-8d33-c2762b0dd755 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:10.539590 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681983b2-d3c0-4878-8d33-c2762b0dd755 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:10.554549 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 187/379] 10.222.0.33 () {56 vars in 1376 bytes} [Sun Sep 11 14:54:10 2022] GET /identity/v3/role_assignments?group.id=937e7e73e3f3416a83b83b64423e5bea&scope.project.id=8f8cdaafdd52411bb20efe51ddc2565c&role.id=d1818eb7f2bd467fbf75bad5cc8e0aea => generated 260 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:10.679799 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51daf012-55f2-4123-8782-9dfaddcb678d None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:10.727701 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef277e72-5bdc-4909-9dd7-28ae0a4bc9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:10.728295 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef277e72-5bdc-4909-9dd7-28ae0a4bc9be None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:10.728568 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef277e72-5bdc-4909-9dd7-28ae0a4bc9be None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:10.728836 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef277e72-5bdc-4909-9dd7-28ae0a4bc9be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:10.729568 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef277e72-5bdc-4909-9dd7-28ae0a4bc9be None admin] RBAC: Adding query filter params (group.id=b87501c1d6954becaf4c80c640bad653, role.id=99e8fe7df8e7446680a81d4eaaa1cb68, scope.project.id=b748811b1c8a461188c96c2e314bf507) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:10.729868 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef277e72-5bdc-4909-9dd7-28ae0a4bc9be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:10.731351 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef277e72-5bdc-4909-9dd7-28ae0a4bc9be None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:10.740324 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 193/380] 10.222.0.33 () {56 vars in 1376 bytes} [Sun Sep 11 14:54:10 2022] GET /identity/v3/role_assignments?group.id=b87501c1d6954becaf4c80c640bad653&scope.project.id=b748811b1c8a461188c96c2e314bf507&role.id=99e8fe7df8e7446680a81d4eaaa1cb68 => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:54:11.045059 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd1212-1b67-45ae-9a7c-41148f3c343d None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:11.045583 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd1212-1b67-45ae-9a7c-41148f3c343d None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:11.045583 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd1212-1b67-45ae-9a7c-41148f3c343d None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:11.047912 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 188/381] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:11.054391 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ce215-0e7d-4711-8e17-402b944bbff5 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:11.054683 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ce215-0e7d-4711-8e17-402b944bbff5 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:11.054993 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ce215-0e7d-4711-8e17-402b944bbff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:11.061716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:11.076877 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-445ce215-0e7d-4711-8e17-402b944bbff5 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:11.078677 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-445ce215-0e7d-4711-8e17-402b944bbff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:11.169467 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 194/382] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:11.557155 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd1212-1b67-45ae-9a7c-41148f3c343d None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:11.569554 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:11.683713 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd1212-1b67-45ae-9a7c-41148f3c343d None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:11.730571 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e7fd8-fcb1-4ebd-b7ab-154646637e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:11.731289 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e7fd8-fcb1-4ebd-b7ab-154646637e48 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:11.731571 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e7fd8-fcb1-4ebd-b7ab-154646637e48 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:11.731840 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e7fd8-fcb1-4ebd-b7ab-154646637e48 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:11.743545 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e7fd8-fcb1-4ebd-b7ab-154646637e48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:11.748535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-968e7fd8-fcb1-4ebd-b7ab-154646637e48 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:54:11.748535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:11.748535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:11.748535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:11.748535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:11.748535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:11.748535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:11.748535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:11.748535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:11.748535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:11.748535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:11.748535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:11.748535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:11.748535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:11.748535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:11.748535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:11.749412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:11.749412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:11.749412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:11.749412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:11.749412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:11.749412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:11.749412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:11.749412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:11.749412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:11.749412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:11.749412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:11.749412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:11.749412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:11.749412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:11.749412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:11.749412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:11.750736 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:11.750736 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:11.750736 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:11.750736 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:11.750736 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:11.750736 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:11.750736 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:11.750736 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:11.750736 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:11.750736 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:11.750736 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:54:11.750736 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:11.750736 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 189/383] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:54:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:11.755885 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ce215-0e7d-4711-8e17-402b944bbff5 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:11.913291 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ce215-0e7d-4711-8e17-402b944bbff5 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:11.952440 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:11.964393 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11376db9-b34d-436b-ae35-f4db5c0ba0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:11.965017 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11376db9-b34d-436b-ae35-f4db5c0ba0a1 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:11.965297 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11376db9-b34d-436b-ae35-f4db5c0ba0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:11.965572 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11376db9-b34d-436b-ae35-f4db5c0ba0a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:11.966239 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11376db9-b34d-436b-ae35-f4db5c0ba0a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:11.966539 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11376db9-b34d-436b-ae35-f4db5c0ba0a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:11.967956 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11376db9-b34d-436b-ae35-f4db5c0ba0a1 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:11.985551 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 195/384] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 14:54:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:11.992002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e7fd8-fcb1-4ebd-b7ab-154646637e48 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:12.061743 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:12.102161 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e7fd8-fcb1-4ebd-b7ab-154646637e48 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:12.140644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f4cf6-c928-4d89-ac03-a2a975c0ecb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:12.141289 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f4cf6-c928-4d89-ac03-a2a975c0ecb9 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:12.141570 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f4cf6-c928-4d89-ac03-a2a975c0ecb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:12.141841 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f4cf6-c928-4d89-ac03-a2a975c0ecb9 None admin] PATH_INFO: `/v3/groups/937e7e73e3f3416a83b83b64423e5bea` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:12.143002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8f4cf6-c928-4d89-ac03-a2a975c0ecb9 None admin] RBAC: Authorizing `identity:get_group(group_id=937e7e73e3f3416a83b83b64423e5bea)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:12.144475 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8f4cf6-c928-4d89-ac03-a2a975c0ecb9 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:12.146378 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 190/385] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 11 14:54:11 2022] GET /identity/v3/groups/937e7e73e3f3416a83b83b64423e5bea => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:12.151606 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11376db9-b34d-436b-ae35-f4db5c0ba0a1 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:12.204359 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f88abb1-14c6-4574-8344-0a827bb233fe None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:12.204634 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f88abb1-14c6-4574-8344-0a827bb233fe None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:12.204829 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f88abb1-14c6-4574-8344-0a827bb233fe None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:12.206168 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 191/386] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:12.211148 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73443445-0f0c-471a-ae9a-a465764802f7 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:12.211431 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73443445-0f0c-471a-ae9a-a465764802f7 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:12.211707 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73443445-0f0c-471a-ae9a-a465764802f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:12.236853 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-73443445-0f0c-471a-ae9a-a465764802f7 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:12.238060 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-73443445-0f0c-471a-ae9a-a465764802f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:12.316843 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 192/387] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:54:12.331625 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11376db9-b34d-436b-ae35-f4db5c0ba0a1 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:12.381897 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148fb86-fa39-47b1-8d4d-e6996f008928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:12.382553 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148fb86-fa39-47b1-8d4d-e6996f008928 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:12.382850 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148fb86-fa39-47b1-8d4d-e6996f008928 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:12.383130 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148fb86-fa39-47b1-8d4d-e6996f008928 None admin] PATH_INFO: `/v3/projects/3b402af2708b41df849963f81e65797e` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:12.384296 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6148fb86-fa39-47b1-8d4d-e6996f008928 None admin] RBAC: Authorizing `identity:get_project(project_id=3b402af2708b41df849963f81e65797e)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:12.385739 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6148fb86-fa39-47b1-8d4d-e6996f008928 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:12.387698 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 196/388] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:54:12 2022] GET /identity/v3/projects/3b402af2708b41df849963f81e65797e => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:12.393077 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73443445-0f0c-471a-ae9a-a465764802f7 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:12.519280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:12.535539 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73443445-0f0c-471a-ae9a-a465764802f7 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:12.572369 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5ff24-7851-46fa-ba28-cb597bb54f30 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:12.572646 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5ff24-7851-46fa-ba28-cb597bb54f30 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:12.572919 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5ff24-7851-46fa-ba28-cb597bb54f30 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:12.573319 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3cf960-996a-4caf-bf47-26f3c7302f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:12.573992 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3cf960-996a-4caf-bf47-26f3c7302f2e None admin] REQUEST_METHOD: `PUT` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:12.574273 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3cf960-996a-4caf-bf47-26f3c7302f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:12.574373 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 197/389] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:12.574576 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3cf960-996a-4caf-bf47-26f3c7302f2e None admin] PATH_INFO: `/v3/projects/3b402af2708b41df849963f81e65797e/groups/937e7e73e3f3416a83b83b64423e5bea/roles/c9da950c75da4939b8bd649c11015650` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:12.576521 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3cf960-996a-4caf-bf47-26f3c7302f2e None admin] RBAC: Authorizing `identity:create_grant(project_id=3b402af2708b41df849963f81e65797e, group_id=937e7e73e3f3416a83b83b64423e5bea, role_id=c9da950c75da4939b8bd649c11015650)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:12.578221 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3cf960-996a-4caf-bf47-26f3c7302f2e None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:12.584107 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16edcf6-a0e0-4c9d-8f39-a92cbbaa3bf6 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:12.584513 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16edcf6-a0e0-4c9d-8f39-a92cbbaa3bf6 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:12.584754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16edcf6-a0e0-4c9d-8f39-a92cbbaa3bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:12.594834 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 193/390] 10.222.0.33 () {58 vars in 1442 bytes} [Sun Sep 11 14:54:12 2022] PUT /identity/v3/projects/3b402af2708b41df849963f81e65797e/groups/937e7e73e3f3416a83b83b64423e5bea/roles/c9da950c75da4939b8bd649c11015650 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:54:12.623985 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-b16edcf6-a0e0-4c9d-8f39-a92cbbaa3bf6 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:12.625885 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-b16edcf6-a0e0-4c9d-8f39-a92cbbaa3bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:12.722863 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:12.735264 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 198/391] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:12.779979 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3cf960-996a-4caf-bf47-26f3c7302f2e None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:12.894598 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3cf960-996a-4caf-bf47-26f3c7302f2e None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:12.929984 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:12.952439 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088362a-19c5-4e63-9df4-f4da9c710bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:12.953256 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088362a-19c5-4e63-9df4-f4da9c710bdf None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:12.953544 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088362a-19c5-4e63-9df4-f4da9c710bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:12.953820 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088362a-19c5-4e63-9df4-f4da9c710bdf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:12.964653 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7088362a-19c5-4e63-9df4-f4da9c710bdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:12.973787 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-7088362a-19c5-4e63-9df4-f4da9c710bdf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:54:12.973787 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:12.973787 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:12.973787 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:12.973787 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:12.973787 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:12.973787 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:12.973787 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:12.973787 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:12.973787 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:12.973787 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:12.973787 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:12.973787 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:12.973787 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:12.973787 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:12.973787 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:12.974719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:12.974719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:12.974719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:12.974719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:12.974719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:12.974719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:12.974719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:12.974719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:12.974719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:12.974719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:12.974719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:12.974719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:12.974719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:12.974719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:12.974719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:12.974719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:12.975966 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:12.975966 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:12.975966 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:12.975966 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:12.975966 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:12.975966 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:12.975966 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:12.975966 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:12.975966 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:12.975966 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:12.975966 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:54:12.975966 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:12.975966 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 194/392] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 14:54:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:12.982733 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16edcf6-a0e0-4c9d-8f39-a92cbbaa3bf6 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:13.093854 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088362a-19c5-4e63-9df4-f4da9c710bdf None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:13.169292 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16edcf6-a0e0-4c9d-8f39-a92cbbaa3bf6 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:13.205817 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088362a-19c5-4e63-9df4-f4da9c710bdf None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:13.240602 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ff5f5-d8b9-4335-9bc2-8e8def9a4683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:13.241245 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ff5f5-d8b9-4335-9bc2-8e8def9a4683 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:13.241527 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ff5f5-d8b9-4335-9bc2-8e8def9a4683 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:13.241801 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ff5f5-d8b9-4335-9bc2-8e8def9a4683 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:13.242580 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6ff5f5-d8b9-4335-9bc2-8e8def9a4683 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:13.242913 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6ff5f5-d8b9-4335-9bc2-8e8def9a4683 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:13.244597 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6ff5f5-d8b9-4335-9bc2-8e8def9a4683 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:13.245505 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96300e95-26b8-4ce6-bd33-f1100639d1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:13.246213 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96300e95-26b8-4ce6-bd33-f1100639d1f8 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:13.246571 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96300e95-26b8-4ce6-bd33-f1100639d1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:13.247049 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96300e95-26b8-4ce6-bd33-f1100639d1f8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:13.260741 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96300e95-26b8-4ce6-bd33-f1100639d1f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:13.262031 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 199/393] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:54:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:13.267599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-96300e95-26b8-4ce6-bd33-f1100639d1f8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:54:13.267599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:13.267599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:13.267599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:13.267599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:13.267599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:13.267599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:13.267599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:13.267599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:13.267599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:13.267599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:13.267599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:13.267599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:13.267599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:13.267599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:13.267599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:13.268555 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:13.268555 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:13.268555 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:13.268555 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:13.268555 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:13.268555 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:13.268555 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:13.268555 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:13.268555 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:13.268555 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:13.268555 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:13.268555 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:13.268555 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:13.268555 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:13.268555 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:13.268555 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:13.269535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:13.269535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:13.269535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:13.269535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:13.269535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:13.269535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:13.269535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:13.269535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:13.269535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:13.269535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:13.269535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:54:13.269535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:13.270151 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ff5f5-d8b9-4335-9bc2-8e8def9a4683 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:13.270225 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 195/394] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:54:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:13.277028 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96300e95-26b8-4ce6-bd33-f1100639d1f8 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:13.331297 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:13.386341 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:13.406062 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96300e95-26b8-4ce6-bd33-f1100639d1f8 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:13.445647 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ff5f5-d8b9-4335-9bc2-8e8def9a4683 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:13.452358 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d28bfe-53f7-4036-9aba-f960506952e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:13.453933 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d28bfe-53f7-4036-9aba-f960506952e6 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:13.454436 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d28bfe-53f7-4036-9aba-f960506952e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:13.455702 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d28bfe-53f7-4036-9aba-f960506952e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:13.456673 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d28bfe-53f7-4036-9aba-f960506952e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:13.457190 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d28bfe-53f7-4036-9aba-f960506952e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:13.459887 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d28bfe-53f7-4036-9aba-f960506952e6 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:13.476181 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 196/395] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 14:54:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:54:13.476181 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c38e9-201a-45dc-b658-90bad3c97dda None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:13.476181 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c38e9-201a-45dc-b658-90bad3c97dda None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:13.476181 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c38e9-201a-45dc-b658-90bad3c97dda None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:13.476181 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 197/396] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:13.478668 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c38e9-201a-45dc-b658-90bad3c97dda None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:13.500789 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8eac3-38e5-4a3a-a18d-8e79fed27a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:13.501430 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8eac3-38e5-4a3a-a18d-8e79fed27a71 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:13.501707 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8eac3-38e5-4a3a-a18d-8e79fed27a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:13.501981 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8eac3-38e5-4a3a-a18d-8e79fed27a71 None admin] PATH_INFO: `/v3/groups/937e7e73e3f3416a83b83b64423e5bea` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:13.503840 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d8eac3-38e5-4a3a-a18d-8e79fed27a71 None admin] RBAC: Authorizing `identity:get_group(group_id=937e7e73e3f3416a83b83b64423e5bea)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:13.505293 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d8eac3-38e5-4a3a-a18d-8e79fed27a71 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:13.507054 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 200/397] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 11 14:54:13 2022] GET /identity/v3/groups/937e7e73e3f3416a83b83b64423e5bea => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:54:13.509146 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891ed93-f0ce-4544-8836-a616a65e2716 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:13.509626 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891ed93-f0ce-4544-8836-a616a65e2716 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:13.510093 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891ed93-f0ce-4544-8836-a616a65e2716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:13.535210 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-0891ed93-f0ce-4544-8836-a616a65e2716 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:13.536822 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-0891ed93-f0ce-4544-8836-a616a65e2716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:13.613239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c38e9-201a-45dc-b658-90bad3c97dda None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:13.622866 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 201/398] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:13.622866 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891ed93-f0ce-4544-8836-a616a65e2716 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:13.631952 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:13.661499 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b962c-7931-472d-a6d5-b29576ac8239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:13.662326 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b962c-7931-472d-a6d5-b29576ac8239 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:13.662860 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b962c-7931-472d-a6d5-b29576ac8239 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:13.663404 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b962c-7931-472d-a6d5-b29576ac8239 None admin] PATH_INFO: `/v3/groups/937e7e73e3f3416a83b83b64423e5bea` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:13.664995 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6b962c-7931-472d-a6d5-b29576ac8239 None admin] RBAC: Authorizing `identity:get_group(group_id=937e7e73e3f3416a83b83b64423e5bea)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:13.666985 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6b962c-7931-472d-a6d5-b29576ac8239 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:13.673794 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:13.674285 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 198/399] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 11 14:54:13 2022] GET /identity/v3/groups/937e7e73e3f3416a83b83b64423e5bea => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:13.679189 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04734b55-8949-4245-8350-019a6225d155 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:13.679755 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04734b55-8949-4245-8350-019a6225d155 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:13.680255 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04734b55-8949-4245-8350-019a6225d155 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:13.683249 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 199/400] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:13.684239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04734b55-8949-4245-8350-019a6225d155 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:13.730677 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891ed93-f0ce-4544-8836-a616a65e2716 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:13.779614 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99fa293-d793-4235-856e-f04c0505c338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:13.781368 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fa293-d793-4235-856e-f04c0505c338 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:13.781882 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fa293-d793-4235-856e-f04c0505c338 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:13.782686 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fa293-d793-4235-856e-f04c0505c338 None admin] PATH_INFO: `/v3/projects/3b402af2708b41df849963f81e65797e` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:13.784839 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99fa293-d793-4235-856e-f04c0505c338 None admin] RBAC: Authorizing `identity:get_project(project_id=3b402af2708b41df849963f81e65797e)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:13.787197 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99fa293-d793-4235-856e-f04c0505c338 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:13.789154 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 202/401] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:54:13 2022] GET /identity/v3/projects/3b402af2708b41df849963f81e65797e => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:13.794541 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c5d81-7130-410d-a421-3a7b606ab2cf None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:13.794541 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c5d81-7130-410d-a421-3a7b606ab2cf None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:13.794541 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c5d81-7130-410d-a421-3a7b606ab2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:13.816780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04734b55-8949-4245-8350-019a6225d155 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:13.817705 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-878c5d81-7130-410d-a421-3a7b606ab2cf None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:13.818909 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-878c5d81-7130-410d-a421-3a7b606ab2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:13.867791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf4943-669c-4813-9ce6-1a588954f34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:13.869326 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf4943-669c-4813-9ce6-1a588954f34c None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:13.869918 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf4943-669c-4813-9ce6-1a588954f34c None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:13.870409 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf4943-669c-4813-9ce6-1a588954f34c None admin] PATH_INFO: `/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:13.872506 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf4943-669c-4813-9ce6-1a588954f34c None admin] RBAC: Authorizing `identity:get_project(project_id=8f8cdaafdd52411bb20efe51ddc2565c)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:13.874740 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf4943-669c-4813-9ce6-1a588954f34c None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:13.882163 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 200/402] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:54:13 2022] GET /identity/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:13.882163 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf4943-669c-4813-9ce6-1a588954f34c None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:13.903010 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 203/403] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:13.904286 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c5d81-7130-410d-a421-3a7b606ab2cf None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:13.936768 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:13.959938 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:14.000460 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf4943-669c-4813-9ce6-1a588954f34c None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:14.031708 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c5d81-7130-410d-a421-3a7b606ab2cf None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:14.045893 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e93f96-95a7-416b-b9c7-186936d6b8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:14.048055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e93f96-95a7-416b-b9c7-186936d6b8a3 None admin] REQUEST_METHOD: `PUT` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:14.048824 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e93f96-95a7-416b-b9c7-186936d6b8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:14.049443 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e93f96-95a7-416b-b9c7-186936d6b8a3 None admin] PATH_INFO: `/v3/projects/3b402af2708b41df849963f81e65797e/groups/937e7e73e3f3416a83b83b64423e5bea/roles/d1818eb7f2bd467fbf75bad5cc8e0aea` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:14.055810 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e93f96-95a7-416b-b9c7-186936d6b8a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=3b402af2708b41df849963f81e65797e, group_id=937e7e73e3f3416a83b83b64423e5bea, role_id=d1818eb7f2bd467fbf75bad5cc8e0aea)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:14.058795 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e93f96-95a7-416b-b9c7-186936d6b8a3 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:14.069089 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69347351-7efd-4b75-86c4-4ca45b2e5045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:14.069745 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69347351-7efd-4b75-86c4-4ca45b2e5045 None admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:14.070018 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69347351-7efd-4b75-86c4-4ca45b2e5045 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:14.070293 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69347351-7efd-4b75-86c4-4ca45b2e5045 None admin] PATH_INFO: `/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c/groups/937e7e73e3f3416a83b83b64423e5bea/roles/c9da950c75da4939b8bd649c11015650` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:14.072628 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69347351-7efd-4b75-86c4-4ca45b2e5045 None admin] RBAC: Authorizing `identity:create_grant(project_id=8f8cdaafdd52411bb20efe51ddc2565c, group_id=937e7e73e3f3416a83b83b64423e5bea, role_id=c9da950c75da4939b8bd649c11015650)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:14.074106 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69347351-7efd-4b75-86c4-4ca45b2e5045 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:14.079099 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 201/404] 10.222.0.33 () {58 vars in 1442 bytes} [Sun Sep 11 14:54:13 2022] PUT /identity/v3/projects/3b402af2708b41df849963f81e65797e/groups/937e7e73e3f3416a83b83b64423e5bea/roles/d1818eb7f2bd467fbf75bad5cc8e0aea => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:54:14.080528 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e93f96-95a7-416b-b9c7-186936d6b8a3 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:14.100091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 204/405] 10.222.0.33 () {58 vars in 1442 bytes} [Sun Sep 11 14:54:13 2022] PUT /identity/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c/groups/937e7e73e3f3416a83b83b64423e5bea/roles/c9da950c75da4939b8bd649c11015650 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:54:14.216521 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69347351-7efd-4b75-86c4-4ca45b2e5045 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:14.236079 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e93f96-95a7-416b-b9c7-186936d6b8a3 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:14.245416 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:14.282796 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516f5a4-0e10-4156-8c06-17b9fe0e79c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:14.283598 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516f5a4-0e10-4156-8c06-17b9fe0e79c1 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:14.283973 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516f5a4-0e10-4156-8c06-17b9fe0e79c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:14.284323 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516f5a4-0e10-4156-8c06-17b9fe0e79c1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:14.300197 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e516f5a4-0e10-4156-8c06-17b9fe0e79c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:14.306287 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-e516f5a4-0e10-4156-8c06-17b9fe0e79c1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:54:14.306287 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:14.306287 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:14.306287 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:14.306287 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:14.306287 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:14.306287 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:14.306287 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:14.306287 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:14.306287 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:14.306287 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:14.306287 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:14.306287 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:14.306287 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:14.306287 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:14.306287 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:14.307296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:14.307296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:14.307296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:14.307296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:14.307296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:14.307296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:14.307296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:14.307296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:14.307296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:14.307296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:14.307296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:14.307296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:14.307296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:14.307296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:14.307296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:14.307296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:14.308348 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:14.308348 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:14.308348 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:14.308348 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:14.308348 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:14.308348 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:14.308348 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:14.308348 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:14.308348 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:14.308348 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:14.308348 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:54:14.308348 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:14.308348 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 202/406] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 14:54:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:14.317800 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516f5a4-0e10-4156-8c06-17b9fe0e79c1 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:14.394030 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69347351-7efd-4b75-86c4-4ca45b2e5045 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:14.409477 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:14.444421 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5175505-8a86-489d-876e-2ad441d20595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:14.445395 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5175505-8a86-489d-876e-2ad441d20595 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:14.445862 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5175505-8a86-489d-876e-2ad441d20595 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:14.446323 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5175505-8a86-489d-876e-2ad441d20595 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:14.459132 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5175505-8a86-489d-876e-2ad441d20595 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:14.465216 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-e5175505-8a86-489d-876e-2ad441d20595 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:54:14.465216 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:14.465216 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:14.465216 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:14.465216 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:14.465216 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:14.465216 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:14.465216 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:14.465216 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:14.465216 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:14.465216 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:14.465216 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:14.465216 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:14.465216 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:14.465216 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:14.465216 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:14.466188 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:14.466188 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:14.466188 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:14.466188 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:14.466188 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:14.466188 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:14.466188 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:14.466188 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:14.466188 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:14.466188 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:14.466188 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:14.466188 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:14.466188 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:14.466188 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:14.466188 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:14.466188 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:14.467074 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:14.467074 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:14.467074 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:14.467074 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:14.467074 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:14.467074 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:14.467074 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:14.467074 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:14.467074 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:14.467074 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:14.467074 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:54:14.467074 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:14.470053 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 205/407] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:54:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:54:14.476305 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5175505-8a86-489d-876e-2ad441d20595 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:14.493988 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516f5a4-0e10-4156-8c06-17b9fe0e79c1 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:14.574559 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a81242-e823-47e9-8593-2746148ac93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:14.575219 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a81242-e823-47e9-8593-2746148ac93d None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:14.575507 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a81242-e823-47e9-8593-2746148ac93d None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:14.575788 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a81242-e823-47e9-8593-2746148ac93d None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:14.576468 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a81242-e823-47e9-8593-2746148ac93d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:14.576776 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a81242-e823-47e9-8593-2746148ac93d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:14.578392 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a81242-e823-47e9-8593-2746148ac93d None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:14.597837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 203/408] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:54:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:14.606240 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a81242-e823-47e9-8593-2746148ac93d None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:14.617973 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5175505-8a86-489d-876e-2ad441d20595 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:14.639489 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:14.712158 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73fca90-80ef-4bbb-b15d-8f62d2461c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:14.712893 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73fca90-80ef-4bbb-b15d-8f62d2461c59 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:14.713063 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73fca90-80ef-4bbb-b15d-8f62d2461c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:14.713318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73fca90-80ef-4bbb-b15d-8f62d2461c59 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:14.713998 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73fca90-80ef-4bbb-b15d-8f62d2461c59 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:14.714278 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73fca90-80ef-4bbb-b15d-8f62d2461c59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:14.715690 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73fca90-80ef-4bbb-b15d-8f62d2461c59 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:14.724210 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a81242-e823-47e9-8593-2746148ac93d None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:14.735290 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 206/409] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:54:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:54:14.742281 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73fca90-80ef-4bbb-b15d-8f62d2461c59 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:14.768863 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6677a7ea-ebfe-4b1f-bd9e-45b022b0832c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:14.769806 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677a7ea-ebfe-4b1f-bd9e-45b022b0832c None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:14.770301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677a7ea-ebfe-4b1f-bd9e-45b022b0832c None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:14.770750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677a7ea-ebfe-4b1f-bd9e-45b022b0832c None admin] PATH_INFO: `/v3/groups/937e7e73e3f3416a83b83b64423e5bea` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:14.772140 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6677a7ea-ebfe-4b1f-bd9e-45b022b0832c None admin] RBAC: Authorizing `identity:get_group(group_id=937e7e73e3f3416a83b83b64423e5bea)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:14.773947 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6677a7ea-ebfe-4b1f-bd9e-45b022b0832c None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:14.775732 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 204/410] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 11 14:54:14 2022] GET /identity/v3/groups/937e7e73e3f3416a83b83b64423e5bea => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:54:14.781343 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6677a7ea-ebfe-4b1f-bd9e-45b022b0832c None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:14.805427 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:14.883598 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:14.891148 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6677a7ea-ebfe-4b1f-bd9e-45b022b0832c None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:14.927409 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2675b-257f-4f4d-9f86-3b7292ed8926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:14.928349 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2675b-257f-4f4d-9f86-3b7292ed8926 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:14.928834 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2675b-257f-4f4d-9f86-3b7292ed8926 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:14.929286 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2675b-257f-4f4d-9f86-3b7292ed8926 None admin] PATH_INFO: `/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:14.930643 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2675b-257f-4f4d-9f86-3b7292ed8926 None admin] RBAC: Authorizing `identity:get_project(project_id=8f8cdaafdd52411bb20efe51ddc2565c)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:14.932286 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2675b-257f-4f4d-9f86-3b7292ed8926 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:14.934436 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 205/411] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:54:14 2022] GET /identity/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:54:14.946503 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2675b-257f-4f4d-9f86-3b7292ed8926 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:14.965830 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73fca90-80ef-4bbb-b15d-8f62d2461c59 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:15.041640 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e5af2-8d6b-47e8-a379-fa252cbb243e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:15.042277 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e5af2-8d6b-47e8-a379-fa252cbb243e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:15.042530 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e5af2-8d6b-47e8-a379-fa252cbb243e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:15.042880 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e5af2-8d6b-47e8-a379-fa252cbb243e None admin] PATH_INFO: `/v3/groups/b87501c1d6954becaf4c80c640bad653` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:15.043981 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8e5af2-8d6b-47e8-a379-fa252cbb243e None admin] RBAC: Authorizing `identity:get_group(group_id=b87501c1d6954becaf4c80c640bad653)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:15.045425 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8e5af2-8d6b-47e8-a379-fa252cbb243e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:15.047087 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 207/412] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 11 14:54:14 2022] GET /identity/v3/groups/b87501c1d6954becaf4c80c640bad653 => generated 229 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:15.055307 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e5af2-8d6b-47e8-a379-fa252cbb243e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:15.085690 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2675b-257f-4f4d-9f86-3b7292ed8926 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:15.150644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3e4a2f-af95-4caf-a95a-44ccaab8fc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:15.151724 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e4a2f-af95-4caf-a95a-44ccaab8fc17 None admin] REQUEST_METHOD: `PUT` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:15.152963 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e4a2f-af95-4caf-a95a-44ccaab8fc17 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:15.153509 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e4a2f-af95-4caf-a95a-44ccaab8fc17 None admin] PATH_INFO: `/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c/groups/937e7e73e3f3416a83b83b64423e5bea/roles/d1818eb7f2bd467fbf75bad5cc8e0aea` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:15.158082 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3e4a2f-af95-4caf-a95a-44ccaab8fc17 None admin] RBAC: Authorizing `identity:create_grant(project_id=8f8cdaafdd52411bb20efe51ddc2565c, group_id=937e7e73e3f3416a83b83b64423e5bea, role_id=d1818eb7f2bd467fbf75bad5cc8e0aea)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:15.162685 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3e4a2f-af95-4caf-a95a-44ccaab8fc17 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:15.186927 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 206/413] 10.222.0.33 () {58 vars in 1442 bytes} [Sun Sep 11 14:54:14 2022] PUT /identity/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c/groups/937e7e73e3f3416a83b83b64423e5bea/roles/d1818eb7f2bd467fbf75bad5cc8e0aea => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 14:54:15.240110 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:15.246582 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e5af2-8d6b-47e8-a379-fa252cbb243e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:15.292208 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adf8e1a-f1f1-40e7-a228-b7da159eca9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:15.292920 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf8e1a-f1f1-40e7-a228-b7da159eca9a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:15.293199 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf8e1a-f1f1-40e7-a228-b7da159eca9a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:15.293473 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf8e1a-f1f1-40e7-a228-b7da159eca9a None admin] PATH_INFO: `/v3/projects/b748811b1c8a461188c96c2e314bf507` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:15.294646 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adf8e1a-f1f1-40e7-a228-b7da159eca9a None admin] RBAC: Authorizing `identity:get_project(project_id=b748811b1c8a461188c96c2e314bf507)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:15.296126 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adf8e1a-f1f1-40e7-a228-b7da159eca9a None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:15.298214 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 208/414] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:54:15 2022] GET /identity/v3/projects/b748811b1c8a461188c96c2e314bf507 => generated 344 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:15.304043 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3e4a2f-af95-4caf-a95a-44ccaab8fc17 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:15.338027 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:15.340270 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60d6cd-71ad-4aed-a269-c26b34fa685e None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:15.340555 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60d6cd-71ad-4aed-a269-c26b34fa685e None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:15.340838 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60d6cd-71ad-4aed-a269-c26b34fa685e None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:15.342452 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 209/415] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:15.349077 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ceff3d-2678-4204-808a-c8d9fa263967 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:15.349432 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ceff3d-2678-4204-808a-c8d9fa263967 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:15.349810 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ceff3d-2678-4204-808a-c8d9fa263967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:15.371008 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-78ceff3d-2678-4204-808a-c8d9fa263967 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:15.372393 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-78ceff3d-2678-4204-808a-c8d9fa263967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:15.472908 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 210/416] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:15.478818 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3e4a2f-af95-4caf-a95a-44ccaab8fc17 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:15.516317 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27ad845-4902-462b-934e-2cd3e6730e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:15.517312 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ad845-4902-462b-934e-2cd3e6730e9d None admin] REQUEST_METHOD: `PUT` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:15.517796 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ad845-4902-462b-934e-2cd3e6730e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:15.518290 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ad845-4902-462b-934e-2cd3e6730e9d None admin] PATH_INFO: `/v3/projects/b748811b1c8a461188c96c2e314bf507/groups/b87501c1d6954becaf4c80c640bad653/roles/99e8fe7df8e7446680a81d4eaaa1cb68` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:15.520946 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27ad845-4902-462b-934e-2cd3e6730e9d None admin] RBAC: Authorizing `identity:create_grant(project_id=b748811b1c8a461188c96c2e314bf507, group_id=b87501c1d6954becaf4c80c640bad653, role_id=99e8fe7df8e7446680a81d4eaaa1cb68)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:15.523412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27ad845-4902-462b-934e-2cd3e6730e9d None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:15.542365 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 207/417] 10.222.0.33 () {58 vars in 1442 bytes} [Sun Sep 11 14:54:15 2022] PUT /identity/v3/projects/b748811b1c8a461188c96c2e314bf507/groups/b87501c1d6954becaf4c80c640bad653/roles/99e8fe7df8e7446680a81d4eaaa1cb68 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 14:54:15.839148 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ceff3d-2678-4204-808a-c8d9fa263967 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:15.963096 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:15.970443 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ceff3d-2678-4204-808a-c8d9fa263967 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:16.051533 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f336ad3-fdbf-44ff-8d3c-3d7e85b03282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:16.052257 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f336ad3-fdbf-44ff-8d3c-3d7e85b03282 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:16.052480 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f336ad3-fdbf-44ff-8d3c-3d7e85b03282 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:16.058827 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f336ad3-fdbf-44ff-8d3c-3d7e85b03282 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:16.075012 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f336ad3-fdbf-44ff-8d3c-3d7e85b03282 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:16.080901 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-2f336ad3-fdbf-44ff-8d3c-3d7e85b03282 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:54:16.080901 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:16.080901 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:16.080901 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:16.080901 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:16.080901 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:16.080901 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:16.080901 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:16.080901 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:16.080901 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:16.080901 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:16.080901 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:16.080901 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:16.080901 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:16.080901 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:16.080901 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:16.081778 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:16.081778 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:16.081778 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:16.081778 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:16.081778 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:16.081778 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:16.081778 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:16.081778 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:16.081778 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:16.081778 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:16.081778 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:16.081778 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:16.081778 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:16.081778 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:16.081778 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:16.081778 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:16.082576 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:16.082576 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:16.082576 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:16.082576 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:16.082576 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:16.082576 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:16.082576 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:16.082576 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:16.082576 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:16.082576 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:16.082576 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:54:16.082576 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:16.082576 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 211/418] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:54:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:16.087848 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27ad845-4902-462b-934e-2cd3e6730e9d None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:16.244169 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:16.278132 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27ad845-4902-462b-934e-2cd3e6730e9d None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:16.336909 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df7749-88f0-4dba-b42e-d5637de56edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:16.337537 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df7749-88f0-4dba-b42e-d5637de56edb None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:16.337843 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df7749-88f0-4dba-b42e-d5637de56edb None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:16.338121 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df7749-88f0-4dba-b42e-d5637de56edb None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:16.338794 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0df7749-88f0-4dba-b42e-d5637de56edb None admin] RBAC: Adding query filter params (name=member) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:16.339195 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0df7749-88f0-4dba-b42e-d5637de56edb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:16.340693 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0df7749-88f0-4dba-b42e-d5637de56edb None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:16.358395 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 208/419] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 14:54:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:16.364410 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f336ad3-fdbf-44ff-8d3c-3d7e85b03282 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:16.546897 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f336ad3-fdbf-44ff-8d3c-3d7e85b03282 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:16.614626 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e492e-fc58-4a12-a706-558dc6065216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:16.615291 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e492e-fc58-4a12-a706-558dc6065216 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:16.615566 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e492e-fc58-4a12-a706-558dc6065216 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:16.615838 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e492e-fc58-4a12-a706-558dc6065216 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:16.668575 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:16.735529 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9e492e-fc58-4a12-a706-558dc6065216 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:16.737022 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9e492e-fc58-4a12-a706-558dc6065216 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:16.747294 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d544c0-394b-41e5-aada-9f45c77567ad None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:16.747587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d544c0-394b-41e5-aada-9f45c77567ad None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:16.747865 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d544c0-394b-41e5-aada-9f45c77567ad None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:16.749195 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 209/420] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:54:16.756786 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302a5e2-45bc-4565-b5ad-4f13c24811dd None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:16.757069 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302a5e2-45bc-4565-b5ad-4f13c24811dd None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:16.757588 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302a5e2-45bc-4565-b5ad-4f13c24811dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:16.801187 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-f302a5e2-45bc-4565-b5ad-4f13c24811dd None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:16.802342 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-f302a5e2-45bc-4565-b5ad-4f13c24811dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:16.966845 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 210/421] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:54:17.007382 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 212/422] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:54:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 643 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:54:17.012727 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302a5e2-45bc-4565-b5ad-4f13c24811dd None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:17.123556 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:17.150420 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302a5e2-45bc-4565-b5ad-4f13c24811dd None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:17.157430 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49986875-4b87-42b0-adb8-a09b5f495cce None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:17.158296 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49986875-4b87-42b0-adb8-a09b5f495cce None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:17.158780 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49986875-4b87-42b0-adb8-a09b5f495cce None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:17.160574 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 213/423] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:17.168178 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab5a896-5bce-40f6-b6b4-9b1a5f4a7df4 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:17.168463 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab5a896-5bce-40f6-b6b4-9b1a5f4a7df4 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:17.168740 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab5a896-5bce-40f6-b6b4-9b1a5f4a7df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:17.205351 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45200b1-ab14-45b6-9b80-88fc604abcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:17.206005 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45200b1-ab14-45b6-9b80-88fc604abcfe None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:17.206286 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45200b1-ab14-45b6-9b80-88fc604abcfe None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:17.206558 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45200b1-ab14-45b6-9b80-88fc604abcfe None admin] PATH_INFO: `/v3/projects/3b402af2708b41df849963f81e65797e` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:17.207974 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45200b1-ab14-45b6-9b80-88fc604abcfe None admin] RBAC: Authorizing `identity:get_project(project_id=3b402af2708b41df849963f81e65797e)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:17.209440 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45200b1-ab14-45b6-9b80-88fc604abcfe None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:17.214425 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 211/424] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:54:17 2022] GET /identity/v3/projects/3b402af2708b41df849963f81e65797e => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:17.215486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-aab5a896-5bce-40f6-b6b4-9b1a5f4a7df4 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:17.216685 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-aab5a896-5bce-40f6-b6b4-9b1a5f4a7df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:17.219640 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45200b1-ab14-45b6-9b80-88fc604abcfe None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:17.328632 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45200b1-ab14-45b6-9b80-88fc604abcfe None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:17.351180 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:17.364027 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6247114-8a6a-468c-bfa5-1f44429e9ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:17.365288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6247114-8a6a-468c-bfa5-1f44429e9ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:17.365762 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6247114-8a6a-468c-bfa5-1f44429e9ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:17.366224 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6247114-8a6a-468c-bfa5-1f44429e9ed5 None admin] PATH_INFO: `/v3/groups/937e7e73e3f3416a83b83b64423e5bea` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:17.367588 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6247114-8a6a-468c-bfa5-1f44429e9ed5 None admin] RBAC: Authorizing `identity:get_group(group_id=937e7e73e3f3416a83b83b64423e5bea)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:17.369218 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6247114-8a6a-468c-bfa5-1f44429e9ed5 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:17.371330 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 212/425] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 11 14:54:17 2022] GET /identity/v3/groups/937e7e73e3f3416a83b83b64423e5bea => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:17.373178 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 214/426] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:17.378862 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6247114-8a6a-468c-bfa5-1f44429e9ed5 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:17.396214 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:17.398491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab5a896-5bce-40f6-b6b4-9b1a5f4a7df4 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:17.523150 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab5a896-5bce-40f6-b6b4-9b1a5f4a7df4 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:17.524445 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6247114-8a6a-468c-bfa5-1f44429e9ed5 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:17.560601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715f3bc-9d2f-4d92-acd9-63b2079b672d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:17.561214 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715f3bc-9d2f-4d92-acd9-63b2079b672d None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:17.561486 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715f3bc-9d2f-4d92-acd9-63b2079b672d None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:17.561847 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715f3bc-9d2f-4d92-acd9-63b2079b672d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:17.562292 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb56e29-781c-4c5a-8d0f-5fd6627d21bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:17.562632 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9715f3bc-9d2f-4d92-acd9-63b2079b672d None admin] RBAC: Adding query filter params (group.id=937e7e73e3f3416a83b83b64423e5bea, role.id=c9da950c75da4939b8bd649c11015650, scope.project.id=3b402af2708b41df849963f81e65797e) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:17.562962 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb56e29-781c-4c5a-8d0f-5fd6627d21bc None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:17.563066 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9715f3bc-9d2f-4d92-acd9-63b2079b672d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:17.563244 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb56e29-781c-4c5a-8d0f-5fd6627d21bc None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:17.563519 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb56e29-781c-4c5a-8d0f-5fd6627d21bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:17.564577 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9715f3bc-9d2f-4d92-acd9-63b2079b672d None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:17.570854 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 213/427] 10.222.0.33 () {56 vars in 1376 bytes} [Sun Sep 11 14:54:17 2022] GET /identity/v3/role_assignments?group.id=937e7e73e3f3416a83b83b64423e5bea&scope.project.id=3b402af2708b41df849963f81e65797e&role.id=c9da950c75da4939b8bd649c11015650 => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:17.600121 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb56e29-781c-4c5a-8d0f-5fd6627d21bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:17.605209 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-6fb56e29-781c-4c5a-8d0f-5fd6627d21bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:54:17.605209 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:17.605209 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:17.605209 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:17.605209 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:17.605209 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:17.605209 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:17.605209 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:17.605209 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:17.605209 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:17.605209 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:17.605209 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:17.605209 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:17.605209 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:17.605209 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:17.605209 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:17.606151 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:17.606151 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:17.606151 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:17.606151 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:17.606151 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:17.606151 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:17.606151 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:17.606151 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:17.606151 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:17.606151 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:17.606151 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:17.606151 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:17.606151 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:17.606151 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:17.606151 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:17.606151 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:17.607194 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:17.607194 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:17.607194 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:17.607194 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:17.607194 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:17.607194 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:17.607194 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:17.607194 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:17.607194 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:17.607194 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:17.607194 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:54:17.607194 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:17.607194 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 215/428] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:54:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:17.614955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715f3bc-9d2f-4d92-acd9-63b2079b672d None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:17.709470 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:17.737952 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715f3bc-9d2f-4d92-acd9-63b2079b672d None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:17.775557 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1853aeb-0e7a-41df-9ff3-44aa4d6e6f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:17.776166 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1853aeb-0e7a-41df-9ff3-44aa4d6e6f2e None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:17.776443 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1853aeb-0e7a-41df-9ff3-44aa4d6e6f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:17.776715 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1853aeb-0e7a-41df-9ff3-44aa4d6e6f2e None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:17.777378 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1853aeb-0e7a-41df-9ff3-44aa4d6e6f2e None admin] RBAC: Adding query filter params (name=member) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:17.777679 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1853aeb-0e7a-41df-9ff3-44aa4d6e6f2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:17.779097 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1853aeb-0e7a-41df-9ff3-44aa4d6e6f2e None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:17.788714 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 214/429] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 14:54:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:17.795180 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb56e29-781c-4c5a-8d0f-5fd6627d21bc None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:17.846223 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1853aeb-0e7a-41df-9ff3-44aa4d6e6f2e None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:17.860617 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:17.935645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb56e29-781c-4c5a-8d0f-5fd6627d21bc None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:17.991433 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e885472-4431-4f78-8b56-96e10019cef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:17.992169 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e885472-4431-4f78-8b56-96e10019cef1 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:17.992457 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e885472-4431-4f78-8b56-96e10019cef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:17.992733 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e885472-4431-4f78-8b56-96e10019cef1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:18.011817 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1853aeb-0e7a-41df-9ff3-44aa4d6e6f2e None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:18.060233 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64da0d-95cd-4386-b8c3-f82daf5193d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:18.060872 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64da0d-95cd-4386-b8c3-f82daf5193d9 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:18.061771 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64da0d-95cd-4386-b8c3-f82daf5193d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:18.062696 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64da0d-95cd-4386-b8c3-f82daf5193d9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:18.065912 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:18.079922 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b64da0d-95cd-4386-b8c3-f82daf5193d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:18.085828 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e885472-4431-4f78-8b56-96e10019cef1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:18.085956 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-3b64da0d-95cd-4386-b8c3-f82daf5193d9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:54:18.085956 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:18.085956 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:18.085956 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:18.085956 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:18.085956 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:18.085956 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:18.085956 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:18.085956 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:18.085956 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:18.085956 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:18.085956 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:18.085956 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:18.085956 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:18.085956 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:18.085956 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:18.086791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:18.086791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:18.086791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:18.086791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:18.086791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:18.086791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:18.086791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:18.086791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:18.086791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:18.086791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:18.086791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:18.086791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:18.086791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:18.086791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:18.086791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:18.086791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:18.087609 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:18.087609 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:18.087609 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:18.087609 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:18.087609 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:18.087609 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:18.087609 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:18.087609 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:18.087609 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:18.087609 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:18.087609 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:54:18.087609 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:18.088228 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e885472-4431-4f78-8b56-96e10019cef1 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:18.088296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 215/430] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 14:54:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:54:18.095463 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64da0d-95cd-4386-b8c3-f82daf5193d9 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:18.160841 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:18.225380 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 216/431] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:54:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:54:18.227036 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ddc59f-5717-4945-9c61-327072a22197 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:18.227520 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ddc59f-5717-4945-9c61-327072a22197 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:18.227983 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ddc59f-5717-4945-9c61-327072a22197 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:18.229651 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 217/432] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:18.232737 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ddc59f-5717-4945-9c61-327072a22197 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:18.245469 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64da0d-95cd-4386-b8c3-f82daf5193d9 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:18.287845 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeaa3db-4563-47fb-a8f9-fb31ed0c0105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:18.288464 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeaa3db-4563-47fb-a8f9-fb31ed0c0105 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:18.288749 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeaa3db-4563-47fb-a8f9-fb31ed0c0105 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:18.289025 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeaa3db-4563-47fb-a8f9-fb31ed0c0105 None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:18.290046 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeaa3db-4563-47fb-a8f9-fb31ed0c0105 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:18.290046 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeaa3db-4563-47fb-a8f9-fb31ed0c0105 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:18.291420 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeaa3db-4563-47fb-a8f9-fb31ed0c0105 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:18.300406 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 216/433] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:54:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:18.304020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598c8fd-3e88-42c8-8943-56558f6c79ab None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:18.304302 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598c8fd-3e88-42c8-8943-56558f6c79ab None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:18.304577 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598c8fd-3e88-42c8-8943-56558f6c79ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:18.323083 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-d598c8fd-3e88-42c8-8943-56558f6c79ab None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:18.324277 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-d598c8fd-3e88-42c8-8943-56558f6c79ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:18.348521 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ddc59f-5717-4945-9c61-327072a22197 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:18.379108 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:18.382443 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c4cbec-a324-4371-ba7e-4a3608fa65a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:18.383110 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4cbec-a324-4371-ba7e-4a3608fa65a2 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:18.383390 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4cbec-a324-4371-ba7e-4a3608fa65a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:18.383663 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4cbec-a324-4371-ba7e-4a3608fa65a2 None admin] PATH_INFO: `/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:18.384758 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c4cbec-a324-4371-ba7e-4a3608fa65a2 None admin] RBAC: Authorizing `identity:get_project(project_id=8f8cdaafdd52411bb20efe51ddc2565c)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:18.386192 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c4cbec-a324-4371-ba7e-4a3608fa65a2 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:18.387862 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 218/434] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:54:18 2022] GET /identity/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:54:18.389040 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a447b31-1bd0-4ce5-89ba-5d814fdd8545 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:18.389323 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a447b31-1bd0-4ce5-89ba-5d814fdd8545 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:18.389590 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a447b31-1bd0-4ce5-89ba-5d814fdd8545 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:18.391308 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 219/435] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:18.391913 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a447b31-1bd0-4ce5-89ba-5d814fdd8545 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:18.397891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 217/436] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:18.398599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d598c8fd-3e88-42c8-8943-56558f6c79ab None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:18.411269 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:18.499131 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a447b31-1bd0-4ce5-89ba-5d814fdd8545 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:18.506231 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d598c8fd-3e88-42c8-8943-56558f6c79ab None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:18.535612 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7672aa94-686f-4262-91d8-f48f69ded199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:18.536226 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672aa94-686f-4262-91d8-f48f69ded199 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:18.536508 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672aa94-686f-4262-91d8-f48f69ded199 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:18.536790 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672aa94-686f-4262-91d8-f48f69ded199 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:18.546964 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b22133-361f-4112-8999-9c782f30f967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:18.548384 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b22133-361f-4112-8999-9c782f30f967 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:18.548878 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b22133-361f-4112-8999-9c782f30f967 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:18.549612 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b22133-361f-4112-8999-9c782f30f967 None admin] PATH_INFO: `/v3/groups/937e7e73e3f3416a83b83b64423e5bea` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:18.551195 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b22133-361f-4112-8999-9c782f30f967 None admin] RBAC: Authorizing `identity:get_group(group_id=937e7e73e3f3416a83b83b64423e5bea)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:18.552777 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b22133-361f-4112-8999-9c782f30f967 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:18.554340 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 218/437] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 11 14:54:18 2022] GET /identity/v3/groups/937e7e73e3f3416a83b83b64423e5bea => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:18.558923 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba943a-283b-4ed6-89a9-78bf93f27b7c None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:18.558923 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba943a-283b-4ed6-89a9-78bf93f27b7c None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:18.558923 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba943a-283b-4ed6-89a9-78bf93f27b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:18.576151 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-abba943a-283b-4ed6-89a9-78bf93f27b7c None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:18.577295 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-abba943a-283b-4ed6-89a9-78bf93f27b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:18.612363 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7672aa94-686f-4262-91d8-f48f69ded199 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:18.613838 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7672aa94-686f-4262-91d8-f48f69ded199 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:18.630711 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:18.637407 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:18.663102 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 219/438] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:18.663838 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba943a-283b-4ed6-89a9-78bf93f27b7c None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:18.718011 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 220/439] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:54:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:54:18.720079 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7672aa94-686f-4262-91d8-f48f69ded199 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:18.772879 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba943a-283b-4ed6-89a9-78bf93f27b7c None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:18.810635 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94702f6-8113-415d-8331-212adc496d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:18.811260 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94702f6-8113-415d-8331-212adc496d07 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:18.811550 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94702f6-8113-415d-8331-212adc496d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:18.811811 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94702f6-8113-415d-8331-212adc496d07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:18.812587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94702f6-8113-415d-8331-212adc496d07 None admin] RBAC: Adding query filter params (group.id=937e7e73e3f3416a83b83b64423e5bea, role.id=c9da950c75da4939b8bd649c11015650, scope.project.id=8f8cdaafdd52411bb20efe51ddc2565c) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:18.812884 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94702f6-8113-415d-8331-212adc496d07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:18.814339 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94702f6-8113-415d-8331-212adc496d07 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:18.820456 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 220/440] 10.222.0.33 () {56 vars in 1376 bytes} [Sun Sep 11 14:54:18 2022] GET /identity/v3/role_assignments?group.id=937e7e73e3f3416a83b83b64423e5bea&scope.project.id=8f8cdaafdd52411bb20efe51ddc2565c&role.id=c9da950c75da4939b8bd649c11015650 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:18.821188 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94702f6-8113-415d-8331-212adc496d07 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:18.842864 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7672aa94-686f-4262-91d8-f48f69ded199 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:18.870414 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:18.886691 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:18.894951 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd82fb-8271-47a7-abcc-0b8f2f5d99fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:18.895772 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd82fb-8271-47a7-abcc-0b8f2f5d99fa None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:18.896146 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd82fb-8271-47a7-abcc-0b8f2f5d99fa None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:18.896520 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd82fb-8271-47a7-abcc-0b8f2f5d99fa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:18.909551 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd82fb-8271-47a7-abcc-0b8f2f5d99fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:18.915163 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-0bfd82fb-8271-47a7-abcc-0b8f2f5d99fa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:54:18.915163 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:18.915163 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:18.915163 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:18.915163 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:18.915163 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:18.915163 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:18.915163 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:18.915163 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:18.915163 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:18.915163 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:18.915163 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:18.915163 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:18.915163 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:18.915163 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:18.915163 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:18.916295 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:18.916295 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:18.916295 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:18.916295 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:18.916295 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:18.916295 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:18.916295 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:18.916295 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:18.916295 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:18.916295 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:18.916295 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:18.916295 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:18.916295 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:18.916295 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:18.916295 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:18.916295 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:18.917280 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:18.917280 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:18.917280 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:18.917280 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:18.917280 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:18.917280 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:18.917280 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:18.917280 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:18.917280 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:18.917280 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:18.917280 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:54:18.917280 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:18.918999 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 221/441] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 14:54:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:18.925865 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd82fb-8271-47a7-abcc-0b8f2f5d99fa None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:18.957713 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94702f6-8113-415d-8331-212adc496d07 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:19.003627 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b706e96d-4e05-4fe9-9edf-609828c50d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:19.004264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706e96d-4e05-4fe9-9edf-609828c50d5a None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:19.004537 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706e96d-4e05-4fe9-9edf-609828c50d5a None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:19.004808 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706e96d-4e05-4fe9-9edf-609828c50d5a None admin] PATH_INFO: `/v3/projects/3b402af2708b41df849963f81e65797e` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:19.005964 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b706e96d-4e05-4fe9-9edf-609828c50d5a None admin] RBAC: Authorizing `identity:get_project(project_id=3b402af2708b41df849963f81e65797e)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:19.007474 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b706e96d-4e05-4fe9-9edf-609828c50d5a None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:19.009098 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 221/442] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:54:18 2022] GET /identity/v3/projects/3b402af2708b41df849963f81e65797e => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:19.009848 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b706e96d-4e05-4fe9-9edf-609828c50d5a None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:19.050410 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd82fb-8271-47a7-abcc-0b8f2f5d99fa None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:19.087674 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c4f668-b9cf-42e6-a38c-8e003d6227c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:19.088277 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4f668-b9cf-42e6-a38c-8e003d6227c4 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:19.088555 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4f668-b9cf-42e6-a38c-8e003d6227c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:19.088822 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4f668-b9cf-42e6-a38c-8e003d6227c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:19.089478 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c4f668-b9cf-42e6-a38c-8e003d6227c4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:19.089776 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c4f668-b9cf-42e6-a38c-8e003d6227c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:19.091237 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c4f668-b9cf-42e6-a38c-8e003d6227c4 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:19.103540 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 222/443] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:54:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:54:19.103540 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c4f668-b9cf-42e6-a38c-8e003d6227c4 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:19.119534 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b706e96d-4e05-4fe9-9edf-609828c50d5a None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:19.137632 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:19.143605 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:19.155162 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8784b0-761e-4bad-92d3-e2e18af48b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:19.156365 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8784b0-761e-4bad-92d3-e2e18af48b41 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:19.156640 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8784b0-761e-4bad-92d3-e2e18af48b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:19.157099 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8784b0-761e-4bad-92d3-e2e18af48b41 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:19.166916 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8784b0-761e-4bad-92d3-e2e18af48b41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:19.172089 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-bd8784b0-761e-4bad-92d3-e2e18af48b41 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:54:19.172089 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:19.172089 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:19.172089 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:19.172089 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:19.172089 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:19.172089 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:19.172089 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:19.172089 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:19.172089 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:19.172089 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:19.172089 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:19.172089 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:19.172089 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:19.172089 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:19.172089 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:19.173179 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:19.173179 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:19.173179 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:19.173179 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:19.173179 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:19.173179 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:19.173179 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:19.173179 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:19.173179 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:19.173179 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:19.173179 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:19.173179 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:19.173179 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:19.173179 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:19.173179 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:19.173179 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:19.174155 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:19.174155 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:19.174155 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:19.174155 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:19.174155 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:19.174155 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:19.174155 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:19.174155 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:19.174155 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:19.174155 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:19.174155 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:54:19.174155 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:19.174155 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 222/444] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:54:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:19.174975 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8784b0-761e-4bad-92d3-e2e18af48b41 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:19.209341 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c4f668-b9cf-42e6-a38c-8e003d6227c4 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:19.244702 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a655f54-7a33-4369-86b2-bbc590c7d9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:19.245335 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a655f54-7a33-4369-86b2-bbc590c7d9c1 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:19.245618 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a655f54-7a33-4369-86b2-bbc590c7d9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:19.245895 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a655f54-7a33-4369-86b2-bbc590c7d9c1 None admin] PATH_INFO: `/v3/groups/937e7e73e3f3416a83b83b64423e5bea` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:19.247159 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a655f54-7a33-4369-86b2-bbc590c7d9c1 None admin] RBAC: Authorizing `identity:get_group(group_id=937e7e73e3f3416a83b83b64423e5bea)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:19.248615 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a655f54-7a33-4369-86b2-bbc590c7d9c1 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:19.250922 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 223/445] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 11 14:54:19 2022] GET /identity/v3/groups/937e7e73e3f3416a83b83b64423e5bea => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:19.251603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a655f54-7a33-4369-86b2-bbc590c7d9c1 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:19.276490 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8784b0-761e-4bad-92d3-e2e18af48b41 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:19.310227 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec1e74b-3d55-418b-8f10-af061f4a8330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:19.310760 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1e74b-3d55-418b-8f10-af061f4a8330 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:19.311074 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1e74b-3d55-418b-8f10-af061f4a8330 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:19.311353 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1e74b-3d55-418b-8f10-af061f4a8330 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:19.355798 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a655f54-7a33-4369-86b2-bbc590c7d9c1 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:19.356824 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:19.363871 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:19.379939 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec1e74b-3d55-418b-8f10-af061f4a8330 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:19.381415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec1e74b-3d55-418b-8f10-af061f4a8330 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:19.390733 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2e0d8-59f8-487b-b000-61d0768000af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:19.391501 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2e0d8-59f8-487b-b000-61d0768000af None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:19.391773 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2e0d8-59f8-487b-b000-61d0768000af None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:19.392039 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2e0d8-59f8-487b-b000-61d0768000af None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:19.392696 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2e0d8-59f8-487b-b000-61d0768000af None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:19.392987 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2e0d8-59f8-487b-b000-61d0768000af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:19.394344 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2e0d8-59f8-487b-b000-61d0768000af None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:19.402283 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 224/446] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:54:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:19.403076 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2e0d8-59f8-487b-b000-61d0768000af None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:19.479615 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 223/447] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:54:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:54:19.480701 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec1e74b-3d55-418b-8f10-af061f4a8330 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:19.506021 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2e0d8-59f8-487b-b000-61d0768000af None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:19.539904 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea854df-d9b6-4aaf-b5bf-4070f6bb21fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:19.540511 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea854df-d9b6-4aaf-b5bf-4070f6bb21fa None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:19.540793 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea854df-d9b6-4aaf-b5bf-4070f6bb21fa None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:19.541227 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea854df-d9b6-4aaf-b5bf-4070f6bb21fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:19.541964 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea854df-d9b6-4aaf-b5bf-4070f6bb21fa None admin] RBAC: Adding query filter params (group.id=937e7e73e3f3416a83b83b64423e5bea, role.id=d1818eb7f2bd467fbf75bad5cc8e0aea, scope.project.id=3b402af2708b41df849963f81e65797e) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:19.542270 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea854df-d9b6-4aaf-b5bf-4070f6bb21fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:19.543748 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea854df-d9b6-4aaf-b5bf-4070f6bb21fa None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:19.551469 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 225/448] 10.222.0.33 () {56 vars in 1376 bytes} [Sun Sep 11 14:54:19 2022] GET /identity/v3/role_assignments?group.id=937e7e73e3f3416a83b83b64423e5bea&scope.project.id=3b402af2708b41df849963f81e65797e&role.id=d1818eb7f2bd467fbf75bad5cc8e0aea => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:19.552266 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea854df-d9b6-4aaf-b5bf-4070f6bb21fa None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:19.568299 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:19.592483 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:19.599870 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec1e74b-3d55-418b-8f10-af061f4a8330 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:19.642137 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa7caa6-9a6f-42a7-b2cf-1acd5be20afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:19.642769 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7caa6-9a6f-42a7-b2cf-1acd5be20afa None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:19.643085 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7caa6-9a6f-42a7-b2cf-1acd5be20afa None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:19.643364 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7caa6-9a6f-42a7-b2cf-1acd5be20afa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:19.674297 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea854df-d9b6-4aaf-b5bf-4070f6bb21fa None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:19.712260 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88beec4-d95e-41e6-855d-4c467ef6c940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:19.712908 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88beec4-d95e-41e6-855d-4c467ef6c940 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:19.713186 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88beec4-d95e-41e6-855d-4c467ef6c940 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:19.713457 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88beec4-d95e-41e6-855d-4c467ef6c940 None admin] PATH_INFO: `/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:19.714717 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88beec4-d95e-41e6-855d-4c467ef6c940 None admin] RBAC: Authorizing `identity:get_project(project_id=8f8cdaafdd52411bb20efe51ddc2565c)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:19.716266 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88beec4-d95e-41e6-855d-4c467ef6c940 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:19.717954 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 226/449] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:54:19 2022] GET /identity/v3/projects/8f8cdaafdd52411bb20efe51ddc2565c => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:19.724151 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88beec4-d95e-41e6-855d-4c467ef6c940 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:19.728171 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa7caa6-9a6f-42a7-b2cf-1acd5be20afa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:19.729661 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa7caa6-9a6f-42a7-b2cf-1acd5be20afa None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:19.821464 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:19.848839 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:19.849438 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88beec4-d95e-41e6-855d-4c467ef6c940 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:19.856896 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 224/450] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:54:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:54:19.858191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa7caa6-9a6f-42a7-b2cf-1acd5be20afa None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:19.892903 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90cdb63-05b7-474e-a3c5-1b4f908679f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:19.893537 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cdb63-05b7-474e-a3c5-1b4f908679f8 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:19.893817 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cdb63-05b7-474e-a3c5-1b4f908679f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:19.894089 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cdb63-05b7-474e-a3c5-1b4f908679f8 None admin] PATH_INFO: `/v3/groups/937e7e73e3f3416a83b83b64423e5bea` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:19.896273 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90cdb63-05b7-474e-a3c5-1b4f908679f8 None admin] RBAC: Authorizing `identity:get_group(group_id=937e7e73e3f3416a83b83b64423e5bea)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:19.897734 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90cdb63-05b7-474e-a3c5-1b4f908679f8 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:19.902891 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 227/451] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 11 14:54:19 2022] GET /identity/v3/groups/937e7e73e3f3416a83b83b64423e5bea => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:54:19.905299 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90cdb63-05b7-474e-a3c5-1b4f908679f8 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:19.970429 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa7caa6-9a6f-42a7-b2cf-1acd5be20afa None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:20.010625 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a50fa9-a613-4d8c-9009-82af45773781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:20.011332 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a50fa9-a613-4d8c-9009-82af45773781 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:20.011612 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a50fa9-a613-4d8c-9009-82af45773781 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:20.011884 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a50fa9-a613-4d8c-9009-82af45773781 None admin] PATH_INFO: `/v3/projects/b748811b1c8a461188c96c2e314bf507` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:20.013154 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a50fa9-a613-4d8c-9009-82af45773781 None admin] RBAC: Authorizing `identity:get_project(project_id=b748811b1c8a461188c96c2e314bf507)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:20.014576 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a50fa9-a613-4d8c-9009-82af45773781 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:20.016863 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 225/452] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:54:19 2022] GET /identity/v3/projects/b748811b1c8a461188c96c2e314bf507 => generated 344 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:54:20.022534 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a50fa9-a613-4d8c-9009-82af45773781 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:20.031328 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90cdb63-05b7-474e-a3c5-1b4f908679f8 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:20.073583 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3975c5d9-af3e-4d83-b25c-c87e1ffbb763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:20.074522 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975c5d9-af3e-4d83-b25c-c87e1ffbb763 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:20.075000 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975c5d9-af3e-4d83-b25c-c87e1ffbb763 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:20.075123 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:20.075217 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975c5d9-af3e-4d83-b25c-c87e1ffbb763 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:20.075909 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3975c5d9-af3e-4d83-b25c-c87e1ffbb763 None admin] RBAC: Adding query filter params (group.id=937e7e73e3f3416a83b83b64423e5bea, role.id=d1818eb7f2bd467fbf75bad5cc8e0aea, scope.project.id=8f8cdaafdd52411bb20efe51ddc2565c) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:20.076124 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3975c5d9-af3e-4d83-b25c-c87e1ffbb763 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:20.078188 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3975c5d9-af3e-4d83-b25c-c87e1ffbb763 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:20.084420 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 228/453] 10.222.0.33 () {56 vars in 1376 bytes} [Sun Sep 11 14:54:19 2022] GET /identity/v3/role_assignments?group.id=937e7e73e3f3416a83b83b64423e5bea&scope.project.id=8f8cdaafdd52411bb20efe51ddc2565c&role.id=d1818eb7f2bd467fbf75bad5cc8e0aea => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:54:20.139360 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a50fa9-a613-4d8c-9009-82af45773781 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:20.176729 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca420956-b260-4815-b668-f2ef18574290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:20.177369 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca420956-b260-4815-b668-f2ef18574290 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:20.177646 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca420956-b260-4815-b668-f2ef18574290 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:20.177915 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca420956-b260-4815-b668-f2ef18574290 None admin] PATH_INFO: `/v3/groups/b87501c1d6954becaf4c80c640bad653` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:20.179125 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca420956-b260-4815-b668-f2ef18574290 None admin] RBAC: Authorizing `identity:get_group(group_id=b87501c1d6954becaf4c80c640bad653)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:20.180599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca420956-b260-4815-b668-f2ef18574290 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:20.182206 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 226/454] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 11 14:54:20 2022] GET /identity/v3/groups/b87501c1d6954becaf4c80c640bad653 => generated 229 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:54:20.189179 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3975c5d9-af3e-4d83-b25c-c87e1ffbb763 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:20.224155 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:20.297294 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3975c5d9-af3e-4d83-b25c-c87e1ffbb763 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:20.332630 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525f632-a010-410f-9874-28af296f457a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:20.333163 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525f632-a010-410f-9874-28af296f457a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:20.333441 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525f632-a010-410f-9874-28af296f457a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:20.333713 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525f632-a010-410f-9874-28af296f457a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:20.334435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7525f632-a010-410f-9874-28af296f457a None admin] RBAC: Adding query filter params (group.id=b87501c1d6954becaf4c80c640bad653, role.id=99e8fe7df8e7446680a81d4eaaa1cb68, scope.project.id=b748811b1c8a461188c96c2e314bf507) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:20.334740 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7525f632-a010-410f-9874-28af296f457a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:20.336228 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7525f632-a010-410f-9874-28af296f457a None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:20.342354 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 229/455] 10.222.0.33 () {56 vars in 1376 bytes} [Sun Sep 11 14:54:20 2022] GET /identity/v3/role_assignments?group.id=b87501c1d6954becaf4c80c640bad653&scope.project.id=b748811b1c8a461188c96c2e314bf507&role.id=99e8fe7df8e7446680a81d4eaaa1cb68 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:22.875150 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee53681-dd2f-4539-8a82-d0851ef89ac4 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:22.876388 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee53681-dd2f-4539-8a82-d0851ef89ac4 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:22.876900 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee53681-dd2f-4539-8a82-d0851ef89ac4 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:22.878726 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 227/456] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:22.884620 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea42eb2-38d4-4c03-ab4d-0b4160400f49 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:22.885474 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea42eb2-38d4-4c03-ab4d-0b4160400f49 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:22.885953 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea42eb2-38d4-4c03-ab4d-0b4160400f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:22.915776 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-0ea42eb2-38d4-4c03-ab4d-0b4160400f49 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:22.917333 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-0ea42eb2-38d4-4c03-ab4d-0b4160400f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:23.036988 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 230/457] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:23.239147 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b26670-e2ba-4b9b-bc7e-fef91fb7000d None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:23.239526 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b26670-e2ba-4b9b-bc7e-fef91fb7000d None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:23.239709 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b26670-e2ba-4b9b-bc7e-fef91fb7000d None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:23.241303 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 228/458] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:23.246269 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3513b-31a0-4470-aa59-608ee75212d9 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:23.246558 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3513b-31a0-4470-aa59-608ee75212d9 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:23.246851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3513b-31a0-4470-aa59-608ee75212d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:23.271783 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-ffe3513b-31a0-4470-aa59-608ee75212d9 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:23.272992 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-ffe3513b-31a0-4470-aa59-608ee75212d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:23.301228 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:23.320423 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0f187-0ef9-4fc3-8ce6-0ef23b77fedc None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:23.321045 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0f187-0ef9-4fc3-8ce6-0ef23b77fedc None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:23.321524 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0f187-0ef9-4fc3-8ce6-0ef23b77fedc None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:23.324395 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 229/459] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:23.331109 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e51b9-c987-431c-b552-53936ff1017d None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:23.331663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e51b9-c987-431c-b552-53936ff1017d None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:23.332141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e51b9-c987-431c-b552-53936ff1017d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:23.363110 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-6a9e51b9-c987-431c-b552-53936ff1017d None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:23.364772 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-6a9e51b9-c987-431c-b552-53936ff1017d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:23.371160 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 231/460] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:23.481066 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 230/461] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:54:23.482910 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe3513b-31a0-4470-aa59-608ee75212d9 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:23.633560 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe3513b-31a0-4470-aa59-608ee75212d9 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:23.682674 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e51b9-c987-431c-b552-53936ff1017d None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:23.683295 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45e999-0627-46cf-849c-a19d3fb00da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:23.683907 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45e999-0627-46cf-849c-a19d3fb00da3 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:23.684190 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45e999-0627-46cf-849c-a19d3fb00da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:23.684464 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45e999-0627-46cf-849c-a19d3fb00da3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:23.696572 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da45e999-0627-46cf-849c-a19d3fb00da3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:23.698882 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da45e999-0627-46cf-849c-a19d3fb00da3 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:23.725253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-da45e999-0627-46cf-849c-a19d3fb00da3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:23.725253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:23.725253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:23.725253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:23.725253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:23.725253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:23.725253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:23.725253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:23.725253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:23.725253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:23.725253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:23.725253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:23.725253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:23.725253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:23.725253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:23.725253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:23.726427 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:23.726427 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:23.726427 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:23.726427 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:23.726427 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:23.726427 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:23.726427 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:23.726427 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:23.726427 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:23.726427 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:23.726427 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:23.726427 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:23.726427 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:23.726427 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:23.726427 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:23.726427 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:23.726427 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:23.727529 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:23.727529 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:23.727529 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:23.728985 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 232/462] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:23.735610 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45e999-0627-46cf-849c-a19d3fb00da3 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:23.766238 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:23.775289 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:23.834839 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e51b9-c987-431c-b552-53936ff1017d None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:23.876216 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0543bb-6919-461a-b9fc-df28d56b658f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:23.877568 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0543bb-6919-461a-b9fc-df28d56b658f None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:23.877994 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0543bb-6919-461a-b9fc-df28d56b658f None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:23.878422 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0543bb-6919-461a-b9fc-df28d56b658f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:23.891755 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0543bb-6919-461a-b9fc-df28d56b658f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:23.893585 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0543bb-6919-461a-b9fc-df28d56b658f None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:23.910822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-7b0543bb-6919-461a-b9fc-df28d56b658f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:23.910822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:23.910822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:23.910822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:23.910822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:23.910822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:23.910822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:23.910822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:23.910822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:23.910822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:23.910822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:23.910822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:23.910822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:23.910822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:23.910822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:23.910822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:23.911854 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:23.911854 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:23.911854 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:23.911854 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:23.911854 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:23.911854 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:23.911854 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:23.911854 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:23.911854 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:23.911854 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:23.911854 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:23.911854 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:23.911854 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:23.911854 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:23.911854 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:23.911854 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:23.911854 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:23.912736 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:23.912736 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:23.912736 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:23.912897 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 231/463] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:54:23.913862 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0543bb-6919-461a-b9fc-df28d56b658f None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:23.935175 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45e999-0627-46cf-849c-a19d3fb00da3 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:23.972045 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021614b5-df3f-4ee5-84d4-016d04d38f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:23.972967 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021614b5-df3f-4ee5-84d4-016d04d38f7d None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:23.973249 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021614b5-df3f-4ee5-84d4-016d04d38f7d None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:23.973787 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021614b5-df3f-4ee5-84d4-016d04d38f7d None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:23.974425 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021614b5-df3f-4ee5-84d4-016d04d38f7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:23.974724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021614b5-df3f-4ee5-84d4-016d04d38f7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:23.977931 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021614b5-df3f-4ee5-84d4-016d04d38f7d None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:23.992050 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 233/464] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:23.993265 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e31ae4-0c73-4cb7-9d54-78e4f95cc73b None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:23.993548 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e31ae4-0c73-4cb7-9d54-78e4f95cc73b None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:23.993819 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e31ae4-0c73-4cb7-9d54-78e4f95cc73b None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:23.995312 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 234/465] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:23.996189 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e31ae4-0c73-4cb7-9d54-78e4f95cc73b None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:24.024528 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:24.031315 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0543bb-6919-461a-b9fc-df28d56b658f None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:24.067258 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56476bc5-1cb9-4d8b-9084-2b7c61dfb63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:24.067881 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56476bc5-1cb9-4d8b-9084-2b7c61dfb63a None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:24.068165 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56476bc5-1cb9-4d8b-9084-2b7c61dfb63a None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:24.068440 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56476bc5-1cb9-4d8b-9084-2b7c61dfb63a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:24.073892 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:24.081080 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56476bc5-1cb9-4d8b-9084-2b7c61dfb63a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:24.082547 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56476bc5-1cb9-4d8b-9084-2b7c61dfb63a None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:24.098018 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-56476bc5-1cb9-4d8b-9084-2b7c61dfb63a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:24.098018 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:24.098018 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:24.098018 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:24.098018 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:24.098018 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:24.098018 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:24.098018 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:24.098018 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:24.098018 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:24.098018 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:24.098018 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:24.098018 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:24.098018 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:24.098018 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:24.098018 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:24.099034 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:24.099034 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:24.099034 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:24.099034 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:24.099034 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:24.099034 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:24.099034 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:24.099034 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:24.099034 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:24.099034 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:24.099034 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:24.099034 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:24.099034 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:24.099034 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:24.099034 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:24.099034 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:24.099034 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:24.104867 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:24.104867 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:24.104867 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:24.104867 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 232/466] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:24.104867 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56476bc5-1cb9-4d8b-9084-2b7c61dfb63a None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:24.120333 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e31ae4-0c73-4cb7-9d54-78e4f95cc73b None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:24.159618 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab1968a-a9b8-410b-96c4-ec26e57a8af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:24.160157 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1968a-a9b8-410b-96c4-ec26e57a8af6 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:24.160431 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1968a-a9b8-410b-96c4-ec26e57a8af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:24.160745 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1968a-a9b8-410b-96c4-ec26e57a8af6 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:24.161386 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab1968a-a9b8-410b-96c4-ec26e57a8af6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:24.161731 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab1968a-a9b8-410b-96c4-ec26e57a8af6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:24.163151 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab1968a-a9b8-410b-96c4-ec26e57a8af6 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:24.178863 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 235/467] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:24.178863 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c58669-5460-412f-83c5-ec6c89e51f08 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:24.178863 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c58669-5460-412f-83c5-ec6c89e51f08 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:24.178863 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c58669-5460-412f-83c5-ec6c89e51f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:24.197508 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-d6c58669-5460-412f-83c5-ec6c89e51f08 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:24.198823 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-d6c58669-5460-412f-83c5-ec6c89e51f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:24.217564 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56476bc5-1cb9-4d8b-9084-2b7c61dfb63a None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:24.251743 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec7d62-9e1f-44f6-a8e8-9e72bfdc6424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:24.252349 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec7d62-9e1f-44f6-a8e8-9e72bfdc6424 None admin] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:24.252628 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec7d62-9e1f-44f6-a8e8-9e72bfdc6424 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:24.252903 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec7d62-9e1f-44f6-a8e8-9e72bfdc6424 None admin] PATH_INFO: `/v3/users` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:24.253546 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeec7d62-9e1f-44f6-a8e8-9e72bfdc6424 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:24.255065 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeec7d62-9e1f-44f6-a8e8-9e72bfdc6424 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:24.269586 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 236/468] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:24.270467 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c58669-5460-412f-83c5-ec6c89e51f08 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:24.295224 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 233/469] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 14:54:24 2022] POST /identity/v3/users => generated 246 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:54:24.302856 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec7d62-9e1f-44f6-a8e8-9e72bfdc6424 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:24.305510 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:24.418884 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:24.499938 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c58669-5460-412f-83c5-ec6c89e51f08 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:24.513577 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec7d62-9e1f-44f6-a8e8-9e72bfdc6424 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:24.552658 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2875e8c4-7c3e-49af-ae72-0376d0f63337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:24.553276 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875e8c4-7c3e-49af-ae72-0376d0f63337 None admin] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:24.553554 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875e8c4-7c3e-49af-ae72-0376d0f63337 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:24.553824 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875e8c4-7c3e-49af-ae72-0376d0f63337 None admin] PATH_INFO: `/v3/users` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:24.554428 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d14ff-7828-4a6b-b4cf-d6d63da70331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:24.554567 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2875e8c4-7c3e-49af-ae72-0376d0f63337 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:24.555077 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d14ff-7828-4a6b-b4cf-d6d63da70331 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:24.555359 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d14ff-7828-4a6b-b4cf-d6d63da70331 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:24.555635 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d14ff-7828-4a6b-b4cf-d6d63da70331 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:24.555946 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2875e8c4-7c3e-49af-ae72-0376d0f63337 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:24.556288 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372d14ff-7828-4a6b-b4cf-d6d63da70331 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:24.556592 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372d14ff-7828-4a6b-b4cf-d6d63da70331 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:24.558231 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372d14ff-7828-4a6b-b4cf-d6d63da70331 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:24.574256 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 237/470] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:24.575459 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d14ff-7828-4a6b-b4cf-d6d63da70331 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:24.581492 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 234/471] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 14:54:24 2022] POST /identity/v3/users => generated 248 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:54:24.582405 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2875e8c4-7c3e-49af-ae72-0376d0f63337 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:24.669722 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:24.711020 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d14ff-7828-4a6b-b4cf-d6d63da70331 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:24.731568 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2875e8c4-7c3e-49af-ae72-0376d0f63337 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:24.736144 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:24.761313 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f054b-4e6c-42d6-a522-3bc1bdc929c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:24.763245 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f054b-4e6c-42d6-a522-3bc1bdc929c9 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:24.763245 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f054b-4e6c-42d6-a522-3bc1bdc929c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:24.763245 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f054b-4e6c-42d6-a522-3bc1bdc929c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:24.773500 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e13d4-f6fb-449d-b770-6c06c61c0328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:24.774209 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e13d4-f6fb-449d-b770-6c06c61c0328 None admin] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:24.774564 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e13d4-f6fb-449d-b770-6c06c61c0328 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:24.774947 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e13d4-f6fb-449d-b770-6c06c61c0328 None admin] PATH_INFO: `/v3/users` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:24.775678 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e13d4-f6fb-449d-b770-6c06c61c0328 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:24.776515 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f054b-4e6c-42d6-a522-3bc1bdc929c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:24.777208 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e13d4-f6fb-449d-b770-6c06c61c0328 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:24.777983 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f054b-4e6c-42d6-a522-3bc1bdc929c9 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:24.796627 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-d06f054b-4e6c-42d6-a522-3bc1bdc929c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:24.796627 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:24.796627 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:24.796627 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:24.796627 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:24.796627 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:24.796627 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:24.796627 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:24.796627 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:24.796627 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:24.796627 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:24.796627 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:24.796627 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:24.796627 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:24.796627 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:24.796627 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:24.797738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:24.797738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:24.797738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:24.797738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:24.797738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:24.797738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:24.797738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:24.797738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:24.797738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:24.797738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:24.797738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:24.797738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:24.797738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:24.797738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:24.797738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:24.797738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:24.797738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:24.798582 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:24.798582 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:24.798582 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:24.798582 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 238/472] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:24.800644 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2e792-1878-4761-a405-7e27434ff5d7 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:24.800644 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2e792-1878-4761-a405-7e27434ff5d7 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:24.800644 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2e792-1878-4761-a405-7e27434ff5d7 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:24.801498 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 239/473] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:24.804347 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b2e792-1878-4761-a405-7e27434ff5d7 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:24.811640 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 235/474] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 14:54:24 2022] POST /identity/v3/users => generated 248 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:54:24.812965 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0035d0-4070-4d42-9f1d-e8c0b6b03347 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:24.813254 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0035d0-4070-4d42-9f1d-e8c0b6b03347 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:24.813532 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0035d0-4070-4d42-9f1d-e8c0b6b03347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:24.846099 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-3c0035d0-4070-4d42-9f1d-e8c0b6b03347 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:24.847730 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-3c0035d0-4070-4d42-9f1d-e8c0b6b03347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:24.939677 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b2e792-1878-4761-a405-7e27434ff5d7 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:24.943495 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 236/475] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:24.982975 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e3d33-cde8-447b-90f4-9813b2c8f529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:24.984197 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e3d33-cde8-447b-90f4-9813b2c8f529 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:24.984583 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e3d33-cde8-447b-90f4-9813b2c8f529 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:24.985159 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e3d33-cde8-447b-90f4-9813b2c8f529 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:24.985908 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e3d33-cde8-447b-90f4-9813b2c8f529 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:24.986213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e3d33-cde8-447b-90f4-9813b2c8f529 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:24.987712 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e3d33-cde8-447b-90f4-9813b2c8f529 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:25.002442 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 240/476] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:25.007923 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0035d0-4070-4d42-9f1d-e8c0b6b03347 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:25.068749 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:25.124897 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0035d0-4070-4d42-9f1d-e8c0b6b03347 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:25.160866 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8a871-162a-445f-85b6-159fe8df7576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:25.161479 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8a871-162a-445f-85b6-159fe8df7576 None admin] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:25.161757 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8a871-162a-445f-85b6-159fe8df7576 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:25.162031 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8a871-162a-445f-85b6-159fe8df7576 None admin] PATH_INFO: `/v3/users` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:25.162679 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc8a871-162a-445f-85b6-159fe8df7576 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:25.164164 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc8a871-162a-445f-85b6-159fe8df7576 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:25.190119 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 237/477] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 14:54:25 2022] POST /identity/v3/users => generated 249 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 14:54:25.256952 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e3d33-cde8-447b-90f4-9813b2c8f529 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:25.295303 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:25.381965 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e3d33-cde8-447b-90f4-9813b2c8f529 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:25.422044 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a5def7-f7a3-4414-8414-2189c127e050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:25.422666 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5def7-f7a3-4414-8414-2189c127e050 None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:25.422976 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5def7-f7a3-4414-8414-2189c127e050 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:25.423247 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5def7-f7a3-4414-8414-2189c127e050 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:25.423917 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a5def7-f7a3-4414-8414-2189c127e050 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:25.425302 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a5def7-f7a3-4414-8414-2189c127e050 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:25.435955 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.sql.core [None req-b4a5def7-f7a3-4414-8414-2189c127e050 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 11 14:54:25.435955 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 11 14:54:25.435955 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [parameters: {'id': 'd588f34f19434b3d9554f0411856875a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 11 14:54:25.435955 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81430) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 11 14:54:25.455037 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-b4a5def7-f7a3-4414-8414-2189c127e050 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 11 14:54:25.455037 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:25.455037 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 14:54:25.455037 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 14:54:25.455037 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 14:54:25.455037 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 14:54:25.455037 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 14:54:25.455037 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 14:54:25.455037 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 14:54:25.455037 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application conn.query(q) Sep 11 14:54:25.455037 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 14:54:25.455037 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 14:54:25.455037 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 14:54:25.455037 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application result.read() Sep 11 14:54:25.455037 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 14:54:25.455949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 14:54:25.455949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 14:54:25.455949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 14:54:25.455949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 14:54:25.455949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 14:54:25.455949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 14:54:25.455949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 14:54:25.455949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 11 14:54:25.455949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:25.455949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 11 14:54:25.455949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:25.455949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:25.455949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 11 14:54:25.455949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 11 14:54:25.455949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 11 14:54:25.455949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return ref.to_dict() Sep 11 14:54:25.455949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 14:54:25.455949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application next(self.gen) Sep 11 14:54:25.456891 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 11 14:54:25.456891 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application yield resource Sep 11 14:54:25.456891 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 14:54:25.456891 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application next(self.gen) Sep 11 14:54:25.456891 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 11 14:54:25.456891 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self.session.rollback() Sep 11 14:54:25.456891 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 11 14:54:25.456891 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self.force_reraise() Sep 11 14:54:25.456891 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 11 14:54:25.456891 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise self.value Sep 11 14:54:25.456891 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 11 14:54:25.456891 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 11 14:54:25.456891 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 11 14:54:25.456891 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application session.commit() Sep 11 14:54:25.456891 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 11 14:54:25.456891 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 11 14:54:25.456891 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 11 14:54:25.457789 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._prepare_impl() Sep 11 14:54:25.457789 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 11 14:54:25.457789 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self.session.flush() Sep 11 14:54:25.457789 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 11 14:54:25.457789 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._flush(objects) Sep 11 14:54:25.457789 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 11 14:54:25.457789 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 11 14:54:25.457789 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 11 14:54:25.457789 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application compat.raise_( Sep 11 14:54:25.457789 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 14:54:25.457789 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception Sep 11 14:54:25.457789 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 11 14:54:25.457789 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application flush_context.execute() Sep 11 14:54:25.457789 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 11 14:54:25.457789 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rec.execute(self) Sep 11 14:54:25.457789 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 11 14:54:25.457789 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 11 14:54:25.458683 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 11 14:54:25.458683 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 11 14:54:25.458683 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 11 14:54:25.458683 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 11 14:54:25.458683 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 11 14:54:25.458683 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 11 14:54:25.458683 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 11 14:54:25.458683 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 11 14:54:25.458683 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 11 14:54:25.458683 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 11 14:54:25.458683 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 11 14:54:25.458683 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 11 14:54:25.458683 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 11 14:54:25.458683 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 11 14:54:25.458683 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 14:54:25.458683 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception Sep 11 14:54:25.461342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 14:54:25.461342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 14:54:25.461342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 14:54:25.461342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 14:54:25.461342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 14:54:25.461342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 14:54:25.461342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 14:54:25.461342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application conn.query(q) Sep 11 14:54:25.461342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 14:54:25.461342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 14:54:25.461342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 14:54:25.461342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application result.read() Sep 11 14:54:25.461342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 14:54:25.461342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 14:54:25.461342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 14:54:25.461342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 14:54:25.462586 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 14:54:25.462586 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 14:54:25.462586 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 14:54:25.462586 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 14:54:25.462586 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 11 14:54:25.462586 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 11 14:54:25.462586 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [parameters: {'id': 'd588f34f19434b3d9554f0411856875a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 11 14:54:25.462586 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 11 14:54:25.462586 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:25.462586 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:54:25.462586 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:25.462586 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:25.462586 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:25.462586 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:25.462586 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:25.462586 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:25.464394 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:25.464394 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:25.464394 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:25.464394 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:25.464394 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:25.464394 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:25.464394 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 11 14:54:25.464394 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 11 14:54:25.464394 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:25.464394 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:25.464394 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 11 14:54:25.464394 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 11 14:54:25.464394 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 11 14:54:25.464394 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 11 14:54:25.464394 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 11 14:54:25.464394 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:25.466326 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 241/478] 10.222.0.33 () {60 vars in 1131 bytes} [Sun Sep 11 14:54:25 2022] POST /identity/v3/roles => generated 144 bytes in 200 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:25.466462 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8a871-162a-445f-85b6-159fe8df7576 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:25.577019 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8a871-162a-445f-85b6-159fe8df7576 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:25.579383 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:25.618329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca909b8-d1bd-4df2-86a0-aca95ef6d0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:25.619010 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca909b8-d1bd-4df2-86a0-aca95ef6d0d4 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:25.619287 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca909b8-d1bd-4df2-86a0-aca95ef6d0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:25.619557 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca909b8-d1bd-4df2-86a0-aca95ef6d0d4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:25.630790 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca909b8-d1bd-4df2-86a0-aca95ef6d0d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:25.636242 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-7ca909b8-d1bd-4df2-86a0-aca95ef6d0d4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:54:25.636242 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:25.636242 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:25.636242 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:25.636242 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:25.636242 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:25.636242 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:25.636242 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:25.636242 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:25.636242 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:25.636242 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:25.636242 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:25.636242 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:25.636242 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:25.636242 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:25.636242 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:25.638268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:25.638268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:25.638268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:25.638268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:25.638268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:25.638268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:25.638268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:25.638268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:25.638268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:25.638268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:25.638268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:25.638268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:25.638268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:25.638268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:25.638268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:25.638268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:25.640748 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:25.640748 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:25.640748 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:25.640748 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:25.640748 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:25.640748 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:25.640748 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:25.640748 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:25.640748 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:25.640748 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:25.640748 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:54:25.640748 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:25.642213 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 238/479] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 14:54:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:25.646918 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a5def7-f7a3-4414-8414-2189c127e050 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:25.749046 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:25.757615 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a5def7-f7a3-4414-8414-2189c127e050 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:25.798628 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22b89d-4a4b-4315-9181-e0d235a89d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:25.799304 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22b89d-4a4b-4315-9181-e0d235a89d7b None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:25.799580 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22b89d-4a4b-4315-9181-e0d235a89d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:25.799849 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22b89d-4a4b-4315-9181-e0d235a89d7b None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:25.800541 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a22b89d-4a4b-4315-9181-e0d235a89d7b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:25.800836 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a22b89d-4a4b-4315-9181-e0d235a89d7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:25.802340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a22b89d-4a4b-4315-9181-e0d235a89d7b None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:25.816246 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 242/480] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:54:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:26.888905 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515fbb6-0310-46fd-a251-9fda093b65f3 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:26.889432 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515fbb6-0310-46fd-a251-9fda093b65f3 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:26.889432 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515fbb6-0310-46fd-a251-9fda093b65f3 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:26.892174 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 239/481] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:26.897314 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28520a-3771-4f40-a0e7-137a23782257 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:26.897778 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28520a-3771-4f40-a0e7-137a23782257 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:26.897855 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28520a-3771-4f40-a0e7-137a23782257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:26.923771 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-dc28520a-3771-4f40-a0e7-137a23782257 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:26.924972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-dc28520a-3771-4f40-a0e7-137a23782257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:27.012894 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 243/482] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:27.199213 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02478249-a567-43f4-b167-7cb992759a57 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:27.199543 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02478249-a567-43f4-b167-7cb992759a57 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:27.199715 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02478249-a567-43f4-b167-7cb992759a57 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:27.201261 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 240/483] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:27.207401 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0e727-96e7-47a2-8b75-d31c71a84d58 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:27.207719 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0e727-96e7-47a2-8b75-d31c71a84d58 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:27.207949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0e727-96e7-47a2-8b75-d31c71a84d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:27.227392 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-60d0e727-96e7-47a2-8b75-d31c71a84d58 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:27.228581 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-60d0e727-96e7-47a2-8b75-d31c71a84d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:27.320109 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 244/484] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:54:27.321584 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02478249-a567-43f4-b167-7cb992759a57 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:27.448453 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02478249-a567-43f4-b167-7cb992759a57 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:27.496870 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc75049-2334-4c6b-b449-6e320d30c1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:27.497513 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc75049-2334-4c6b-b449-6e320d30c1f6 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:27.497800 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc75049-2334-4c6b-b449-6e320d30c1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:27.498075 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc75049-2334-4c6b-b449-6e320d30c1f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:27.512146 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc75049-2334-4c6b-b449-6e320d30c1f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:27.517641 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-3dc75049-2334-4c6b-b449-6e320d30c1f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:54:27.517641 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:27.517641 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:27.517641 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:27.517641 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:27.517641 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:27.517641 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:27.517641 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:27.517641 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:27.517641 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:27.517641 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:27.517641 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:27.517641 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:27.517641 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:27.517641 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:27.517641 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:27.518535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:27.518535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:27.518535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:27.518535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:27.518535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:27.518535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:27.518535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:27.518535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:27.518535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:27.518535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:27.518535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:27.518535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:27.518535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:27.518535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:27.518535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:27.518535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:27.519400 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:27.519400 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:27.519400 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:27.519400 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:27.519400 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:27.519400 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:27.519400 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:27.519400 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:27.519400 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:27.519400 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:27.519400 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:54:27.519400 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:27.519400 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 241/485] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 14:54:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:27.527902 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d0e727-96e7-47a2-8b75-d31c71a84d58 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:27.563665 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:27.690365 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d0e727-96e7-47a2-8b75-d31c71a84d58 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:27.752905 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ca56e-771b-4fce-b053-04096c48ab3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:27.753435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ca56e-771b-4fce-b053-04096c48ab3f None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:27.753715 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ca56e-771b-4fce-b053-04096c48ab3f None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:27.753988 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ca56e-771b-4fce-b053-04096c48ab3f None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:27.754655 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6ca56e-771b-4fce-b053-04096c48ab3f None admin] RBAC: Adding query filter params (name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:27.754992 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6ca56e-771b-4fce-b053-04096c48ab3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:27.756388 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6ca56e-771b-4fce-b053-04096c48ab3f None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:27.771278 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 245/486] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 14:54:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:27.777150 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc75049-2334-4c6b-b449-6e320d30c1f6 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:27.791258 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ca56e-771b-4fce-b053-04096c48ab3f None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:27.820425 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:27.907953 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:27.931057 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ca56e-771b-4fce-b053-04096c48ab3f None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:27.941180 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc75049-2334-4c6b-b449-6e320d30c1f6 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:27.969367 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c059a8-e8ed-4ac9-9177-7ba26330a64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:27.970575 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c059a8-e8ed-4ac9-9177-7ba26330a64e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:27.971085 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c059a8-e8ed-4ac9-9177-7ba26330a64e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:27.971559 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c059a8-e8ed-4ac9-9177-7ba26330a64e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:27.983647 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c059a8-e8ed-4ac9-9177-7ba26330a64e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:27.985871 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a4c36d-1807-4476-8f23-7032160d4bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:27.986508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4c36d-1807-4476-8f23-7032160d4bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:27.986788 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4c36d-1807-4476-8f23-7032160d4bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:27.987089 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4c36d-1807-4476-8f23-7032160d4bf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:27.991426 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-b4c059a8-e8ed-4ac9-9177-7ba26330a64e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:54:27.991426 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:27.991426 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:27.991426 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:27.991426 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:27.991426 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:27.991426 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:27.991426 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:27.991426 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:27.991426 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:27.991426 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:27.991426 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:27.991426 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:27.991426 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:27.991426 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:27.991426 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:27.992317 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:27.992317 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:27.992317 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:27.992317 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:27.992317 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:27.992317 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:27.992317 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:27.992317 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:27.992317 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:27.992317 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:27.992317 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:27.992317 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:27.992317 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:27.992317 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:27.992317 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:27.992317 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:27.993122 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:27.993122 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:27.993122 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:27.993122 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:27.993122 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:27.993122 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:27.993122 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:27.993122 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:27.993122 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:27.993122 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:27.993122 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:54:27.993122 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:27.995792 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 246/487] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 14:54:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:27.997183 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6edf2ad-56c8-4973-8b4b-2568b3e20447 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:27.997744 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6edf2ad-56c8-4973-8b4b-2568b3e20447 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:27.998229 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6edf2ad-56c8-4973-8b4b-2568b3e20447 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:28.005278 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a4c36d-1807-4476-8f23-7032160d4bf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:28.005507 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a4c36d-1807-4476-8f23-7032160d4bf3 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:28.005718 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 247/488] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:28.006926 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6edf2ad-56c8-4973-8b4b-2568b3e20447 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:28.023150 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-c8a4c36d-1807-4476-8f23-7032160d4bf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:28.023150 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:28.023150 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:28.023150 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:28.023150 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:28.023150 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:28.023150 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:28.023150 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:28.023150 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:28.023150 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:28.023150 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:28.023150 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:28.023150 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:28.023150 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:28.023150 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:28.023150 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:28.024090 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:28.024090 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:28.024090 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:28.024090 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:28.024090 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:28.024090 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:28.024090 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:28.024090 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:28.024090 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:28.024090 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:28.024090 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:28.024090 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:28.024090 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:28.024090 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:28.024090 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:28.024090 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:28.024090 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:28.024954 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:28.024954 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:28.024954 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:28.024954 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 242/489] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:28.025730 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811a4ca-b82b-41fe-b671-17c91312584a None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:28.026011 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811a4ca-b82b-41fe-b671-17c91312584a None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:28.026291 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811a4ca-b82b-41fe-b671-17c91312584a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:28.051820 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-f811a4ca-b82b-41fe-b671-17c91312584a None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:28.053338 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-f811a4ca-b82b-41fe-b671-17c91312584a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:28.146842 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 243/490] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:28.146842 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f811a4ca-b82b-41fe-b671-17c91312584a None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:28.146842 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:28.159833 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6edf2ad-56c8-4973-8b4b-2568b3e20447 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:28.248240 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94686b71-4efb-46bf-b05d-916bc7aaaff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:28.248882 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94686b71-4efb-46bf-b05d-916bc7aaaff6 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:28.249316 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94686b71-4efb-46bf-b05d-916bc7aaaff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:28.249572 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94686b71-4efb-46bf-b05d-916bc7aaaff6 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:28.250272 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94686b71-4efb-46bf-b05d-916bc7aaaff6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:28.250571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94686b71-4efb-46bf-b05d-916bc7aaaff6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:28.252037 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94686b71-4efb-46bf-b05d-916bc7aaaff6 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:28.262793 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f811a4ca-b82b-41fe-b671-17c91312584a None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:28.263605 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:28.273218 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 248/491] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 14:54:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:28.289672 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94686b71-4efb-46bf-b05d-916bc7aaaff6 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:28.313382 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee39c62-85c0-4efd-928b-69ec6ede3215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:28.314010 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee39c62-85c0-4efd-928b-69ec6ede3215 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:28.314287 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee39c62-85c0-4efd-928b-69ec6ede3215 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:28.314556 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee39c62-85c0-4efd-928b-69ec6ede3215 None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:28.315247 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee39c62-85c0-4efd-928b-69ec6ede3215 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:28.315550 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee39c62-85c0-4efd-928b-69ec6ede3215 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:28.316965 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee39c62-85c0-4efd-928b-69ec6ede3215 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:28.343977 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 244/492] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:28.351159 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f9111-578d-4fbf-9d8d-f18192ae27f8 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:28.351439 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f9111-578d-4fbf-9d8d-f18192ae27f8 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:28.351714 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f9111-578d-4fbf-9d8d-f18192ae27f8 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:28.353260 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 245/493] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:28.354045 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f9111-578d-4fbf-9d8d-f18192ae27f8 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:28.430150 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94686b71-4efb-46bf-b05d-916bc7aaaff6 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:28.437013 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:28.468796 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f5eca-7ca4-4dbe-8280-f3a749a8ff96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:28.469430 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f5eca-7ca4-4dbe-8280-f3a749a8ff96 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:28.469707 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f5eca-7ca4-4dbe-8280-f3a749a8ff96 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:28.469974 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f5eca-7ca4-4dbe-8280-f3a749a8ff96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:28.482171 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27f5eca-7ca4-4dbe-8280-f3a749a8ff96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:28.484224 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27f5eca-7ca4-4dbe-8280-f3a749a8ff96 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:28.487412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f9111-578d-4fbf-9d8d-f18192ae27f8 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:28.499801 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-f27f5eca-7ca4-4dbe-8280-f3a749a8ff96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:28.499801 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:28.499801 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:28.499801 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:28.499801 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:28.499801 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:28.499801 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:28.499801 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:28.499801 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:28.499801 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:28.499801 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:28.499801 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:28.499801 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:28.499801 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:28.499801 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:28.499801 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:28.500782 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:28.500782 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:28.500782 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:28.500782 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:28.500782 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:28.500782 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:28.500782 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:28.500782 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:28.500782 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:28.500782 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:28.500782 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:28.500782 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:28.500782 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:28.500782 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:28.500782 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:28.500782 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:28.500782 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:28.502296 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:28.502296 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:28.502296 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:28.502296 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 249/494] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:28.502509 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734e950-a546-4035-b56a-f1d66176e8c1 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:28.502758 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734e950-a546-4035-b56a-f1d66176e8c1 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:28.503062 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734e950-a546-4035-b56a-f1d66176e8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:28.521224 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-6734e950-a546-4035-b56a-f1d66176e8c1 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:28.522353 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6734e950-a546-4035-b56a-f1d66176e8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:28.525802 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a60b59d-ef44-4769-88a3-01f16efc19b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:28.525802 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60b59d-ef44-4769-88a3-01f16efc19b4 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:28.526015 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60b59d-ef44-4769-88a3-01f16efc19b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:28.526290 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60b59d-ef44-4769-88a3-01f16efc19b4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:28.534641 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a60b59d-ef44-4769-88a3-01f16efc19b4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:28.536176 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a60b59d-ef44-4769-88a3-01f16efc19b4 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:28.550318 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-3a60b59d-ef44-4769-88a3-01f16efc19b4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 14:54:28.550318 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:28.550318 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:28.550318 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:28.550318 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:28.550318 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:28.550318 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:28.550318 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:28.550318 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:28.550318 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:28.550318 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:28.550318 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:28.550318 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:54:28.550318 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:54:28.550318 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:54:28.550318 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:54:28.551280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:28.551280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:28.551280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:54:28.551280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:28.551280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:54:28.551280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:28.551280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:28.551280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:28.551280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:28.551280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:28.551280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:28.551280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:28.551280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:28.551280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:28.551280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:28.551280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:28.551280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:28.551280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:28.552197 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:28.552197 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:28.552197 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:54:28.552197 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:54:28.552197 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:54:28.552197 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:54:28.552197 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:54:28.552197 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:54:28.552197 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 14:54:28.552197 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:28.552197 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 246/495] 10.222.0.33 () {56 vars in 1093 bytes} [Sun Sep 11 14:54:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:28.553404 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a60b59d-ef44-4769-88a3-01f16efc19b4 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:28.581092 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:28.610860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 250/496] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:28.610860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6734e950-a546-4035-b56a-f1d66176e8c1 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:28.689828 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a60b59d-ef44-4769-88a3-01f16efc19b4 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:28.718589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:28.725451 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6734e950-a546-4035-b56a-f1d66176e8c1 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:28.732379 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589e4cd-ed24-413d-878d-1cc381527367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:28.733459 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589e4cd-ed24-413d-878d-1cc381527367 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:28.733821 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589e4cd-ed24-413d-878d-1cc381527367 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:28.734098 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589e4cd-ed24-413d-878d-1cc381527367 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:28.745657 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589e4cd-ed24-413d-878d-1cc381527367 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:28.751653 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-5589e4cd-ed24-413d-878d-1cc381527367 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:54:28.751653 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:28.751653 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:28.751653 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:28.751653 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:28.751653 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:28.751653 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:28.751653 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:28.751653 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:28.751653 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:28.751653 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:28.751653 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:28.751653 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:28.751653 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:28.751653 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:28.751653 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:28.752545 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:28.752545 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:28.752545 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:28.752545 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:28.752545 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:28.752545 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:28.752545 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:28.752545 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:28.752545 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:28.752545 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:28.752545 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:28.752545 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:28.752545 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:28.752545 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:28.752545 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:28.752545 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:28.753352 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:28.753352 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:28.753352 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:28.753352 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:28.753352 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:28.753352 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:28.753352 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:28.753352 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:28.753352 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:28.753352 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:28.753352 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:54:28.753352 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:28.753352 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 247/497] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 14:54:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:28.754025 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589e4cd-ed24-413d-878d-1cc381527367 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:28.766636 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7bd495-7dcd-4c99-9240-3ecf477aa66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:28.767302 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7bd495-7dcd-4c99-9240-3ecf477aa66f None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:28.767580 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7bd495-7dcd-4c99-9240-3ecf477aa66f None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:28.767850 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7bd495-7dcd-4c99-9240-3ecf477aa66f None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:28.768485 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7bd495-7dcd-4c99-9240-3ecf477aa66f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:28.768781 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7bd495-7dcd-4c99-9240-3ecf477aa66f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:28.770179 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7bd495-7dcd-4c99-9240-3ecf477aa66f None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:28.782594 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 251/498] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:28.783730 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef620e61-ab48-48ed-900c-1205bf1c789d None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:28.784006 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef620e61-ab48-48ed-900c-1205bf1c789d None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:28.784275 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef620e61-ab48-48ed-900c-1205bf1c789d None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:28.789682 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 252/499] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:28.789682 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef620e61-ab48-48ed-900c-1205bf1c789d None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:28.836543 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:28.866294 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589e4cd-ed24-413d-878d-1cc381527367 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:28.899001 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef620e61-ab48-48ed-900c-1205bf1c789d None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:28.902881 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6510bfb0-83dc-469a-867e-1f8524a8b8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:28.903488 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510bfb0-83dc-469a-867e-1f8524a8b8dc None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:28.903768 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510bfb0-83dc-469a-867e-1f8524a8b8dc None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:28.904040 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510bfb0-83dc-469a-867e-1f8524a8b8dc None admin] PATH_INFO: `/v3/users` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:28.904764 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6510bfb0-83dc-469a-867e-1f8524a8b8dc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:28.905065 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6510bfb0-83dc-469a-867e-1f8524a8b8dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:28.906891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6510bfb0-83dc-469a-867e-1f8524a8b8dc None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:28.932209 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e7079-bf95-4a65-a70d-8999f3d72dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:28.933106 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e7079-bf95-4a65-a70d-8999f3d72dcc None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:28.933358 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e7079-bf95-4a65-a70d-8999f3d72dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:28.933632 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e7079-bf95-4a65-a70d-8999f3d72dcc None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:28.934295 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e7079-bf95-4a65-a70d-8999f3d72dcc None admin] RBAC: Adding query filter params (name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:28.934596 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e7079-bf95-4a65-a70d-8999f3d72dcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:28.936017 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e7079-bf95-4a65-a70d-8999f3d72dcc None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:28.944958 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 253/500] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 14:54:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:28.950184 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 248/501] 10.222.0.33 () {56 vars in 1133 bytes} [Sun Sep 11 14:54:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:28.950652 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e7079-bf95-4a65-a70d-8999f3d72dcc None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:28.950743 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de23ca72-3d68-4352-a3db-130df650a5aa None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:28.950743 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de23ca72-3d68-4352-a3db-130df650a5aa None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:28.950743 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de23ca72-3d68-4352-a3db-130df650a5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:28.965312 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-de23ca72-3d68-4352-a3db-130df650a5aa None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:28.966533 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-de23ca72-3d68-4352-a3db-130df650a5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:29.003057 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:29.036098 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 249/502] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:29.036861 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de23ca72-3d68-4352-a3db-130df650a5aa None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:29.054591 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e7079-bf95-4a65-a70d-8999f3d72dcc None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:29.057747 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:29.095882 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e8a47e-9ea0-4f63-bb91-47a34f5581ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:29.096516 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8a47e-9ea0-4f63-bb91-47a34f5581ad None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:29.096799 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8a47e-9ea0-4f63-bb91-47a34f5581ad None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:29.097073 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8a47e-9ea0-4f63-bb91-47a34f5581ad None admin] PATH_INFO: `/v3/users/glance` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:29.104675 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e8a47e-9ea0-4f63-bb91-47a34f5581ad None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:29.106145 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e8a47e-9ea0-4f63-bb91-47a34f5581ad None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:29.119083 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-94e8a47e-9ea0-4f63-bb91-47a34f5581ad None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 11 14:54:29.119083 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:29.119083 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:29.119083 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:29.119083 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:29.119083 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:29.119083 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:29.119083 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:29.119083 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:29.119083 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:29.119083 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:29.119083 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:29.119083 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:54:29.119083 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:54:29.119083 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:54:29.119083 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:54:29.119980 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:29.119980 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:29.119980 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:54:29.119980 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:29.119980 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:54:29.119980 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:29.119980 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:29.119980 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:29.119980 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:29.119980 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:29.119980 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:29.119980 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:29.119980 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:29.119980 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:29.119980 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:29.119980 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:29.119980 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:29.119980 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:29.120962 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:29.120962 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:29.120962 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:54:29.120962 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:54:29.120962 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:54:29.120962 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:54:29.120962 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:54:29.120962 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:54:29.120962 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 11 14:54:29.120962 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:29.120962 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 254/503] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:54:28 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:29.120962 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e8a47e-9ea0-4f63-bb91-47a34f5581ad None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:29.159408 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de23ca72-3d68-4352-a3db-130df650a5aa None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:29.194465 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5578e95-9d04-4972-87c3-808433509aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:29.195132 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5578e95-9d04-4972-87c3-808433509aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:29.195464 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5578e95-9d04-4972-87c3-808433509aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:29.195742 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5578e95-9d04-4972-87c3-808433509aa8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:29.207351 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5578e95-9d04-4972-87c3-808433509aa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:29.212280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-c5578e95-9d04-4972-87c3-808433509aa8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:54:29.212280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:29.212280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:29.212280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:29.212280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:29.212280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:29.212280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:29.212280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:29.212280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:29.212280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:29.212280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:29.212280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:29.212280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:29.212280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:29.212280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:29.212280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:29.213183 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:29.213183 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:29.213183 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:29.213183 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:29.213183 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:29.213183 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:29.213183 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:29.213183 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:29.213183 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:29.213183 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:29.213183 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:29.213183 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:29.213183 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:29.213183 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:29.213183 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:29.213183 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:29.213988 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:29.213988 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:29.213988 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:29.213988 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:29.213988 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:29.213988 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:29.213988 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:29.213988 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:29.213988 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:29.213988 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:29.213988 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:54:29.213988 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:29.213988 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 250/504] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 14:54:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:29.214651 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5578e95-9d04-4972-87c3-808433509aa8 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:29.240441 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e8a47e-9ea0-4f63-bb91-47a34f5581ad None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:29.248777 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:29.284386 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04144150-e71c-4e6f-8e28-c2f268800098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:29.285013 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04144150-e71c-4e6f-8e28-c2f268800098 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:29.285292 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04144150-e71c-4e6f-8e28-c2f268800098 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:29.285563 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04144150-e71c-4e6f-8e28-c2f268800098 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:29.296458 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04144150-e71c-4e6f-8e28-c2f268800098 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:29.297876 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04144150-e71c-4e6f-8e28-c2f268800098 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:29.312821 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-04144150-e71c-4e6f-8e28-c2f268800098 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:29.312821 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:29.312821 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:29.312821 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:29.312821 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:29.312821 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:29.312821 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:29.312821 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:29.312821 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:29.312821 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:29.312821 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:29.312821 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:29.312821 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:29.312821 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:29.312821 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:29.312821 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:29.313859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:29.313859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:29.313859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:29.313859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:29.313859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:29.313859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:29.313859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:29.313859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:29.313859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:29.313859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:29.313859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:29.313859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:29.313859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:29.313859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:29.313859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:29.313859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:29.313859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:29.315406 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:29.315406 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:29.315406 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:29.315406 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 255/505] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:29.315406 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04144150-e71c-4e6f-8e28-c2f268800098 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:29.325259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5578e95-9d04-4972-87c3-808433509aa8 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:29.333266 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:29.360931 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d0430-8d56-475f-857b-c802b52e3b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:29.361534 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d0430-8d56-475f-857b-c802b52e3b23 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:29.361810 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d0430-8d56-475f-857b-c802b52e3b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:29.362079 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d0430-8d56-475f-857b-c802b52e3b23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:29.413400 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04144150-e71c-4e6f-8e28-c2f268800098 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:29.430305 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917d0430-8d56-475f-857b-c802b52e3b23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:29.431858 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917d0430-8d56-475f-857b-c802b52e3b23 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:29.447625 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d7058-30a0-47e1-a7d0-8570836396bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:29.448224 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d7058-30a0-47e1-a7d0-8570836396bd None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:29.448498 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d7058-30a0-47e1-a7d0-8570836396bd None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:29.448772 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d7058-30a0-47e1-a7d0-8570836396bd None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:29.449488 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6d7058-30a0-47e1-a7d0-8570836396bd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:29.449782 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6d7058-30a0-47e1-a7d0-8570836396bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:29.451248 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6d7058-30a0-47e1-a7d0-8570836396bd None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:29.462332 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:29.488596 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 256/506] 10.222.0.33 () {56 vars in 1137 bytes} [Sun Sep 11 14:54:29 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:29.489426 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d7058-30a0-47e1-a7d0-8570836396bd None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:29.533256 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 251/507] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:54:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:54:29.534101 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d0430-8d56-475f-857b-c802b52e3b23 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:29.563594 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:29.588626 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d7058-30a0-47e1-a7d0-8570836396bd None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:29.623757 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6471c90-7d87-41d0-9ffa-8964304fa37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:29.624387 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6471c90-7d87-41d0-9ffa-8964304fa37e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:29.624830 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6471c90-7d87-41d0-9ffa-8964304fa37e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:29.625105 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6471c90-7d87-41d0-9ffa-8964304fa37e None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:29.625760 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6471c90-7d87-41d0-9ffa-8964304fa37e None admin] RBAC: Adding query filter params (name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:29.626054 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6471c90-7d87-41d0-9ffa-8964304fa37e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:29.627476 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6471c90-7d87-41d0-9ffa-8964304fa37e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:29.634564 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d0430-8d56-475f-857b-c802b52e3b23 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:29.635118 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 257/508] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 14:54:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:29.635882 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6471c90-7d87-41d0-9ffa-8964304fa37e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:29.669935 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:29.678619 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf90c88-0027-4dc7-a791-400e6ce97f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:29.679282 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf90c88-0027-4dc7-a791-400e6ce97f20 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:29.679564 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf90c88-0027-4dc7-a791-400e6ce97f20 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:29.679838 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf90c88-0027-4dc7-a791-400e6ce97f20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:29.690198 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf90c88-0027-4dc7-a791-400e6ce97f20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:29.691699 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf90c88-0027-4dc7-a791-400e6ce97f20 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:29.706948 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-dcf90c88-0027-4dc7-a791-400e6ce97f20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:29.706948 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:29.706948 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:29.706948 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:29.706948 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:29.706948 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:29.706948 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:29.706948 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:29.706948 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:29.706948 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:29.706948 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:29.706948 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:29.706948 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:29.706948 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:29.706948 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:29.706948 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:29.707998 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:29.707998 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:29.707998 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:29.707998 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:29.707998 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:29.707998 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:29.707998 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:29.707998 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:29.707998 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:29.707998 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:29.707998 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:29.707998 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:29.707998 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:29.707998 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:29.707998 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:29.707998 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:29.707998 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:29.708932 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:29.708932 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:29.708932 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:29.708932 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 252/509] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:29.709208 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf90c88-0027-4dc7-a791-400e6ce97f20 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:29.734960 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6471c90-7d87-41d0-9ffa-8964304fa37e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:29.768060 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964e2218-1441-4d5e-a9de-a72233668341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:29.768667 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e2218-1441-4d5e-a9de-a72233668341 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:29.768948 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e2218-1441-4d5e-a9de-a72233668341 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:29.769219 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e2218-1441-4d5e-a9de-a72233668341 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:29.769857 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964e2218-1441-4d5e-a9de-a72233668341 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:29.770162 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964e2218-1441-4d5e-a9de-a72233668341 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:29.771652 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964e2218-1441-4d5e-a9de-a72233668341 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:29.773897 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:29.784348 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 258/510] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:29.785152 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964e2218-1441-4d5e-a9de-a72233668341 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:29.809522 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf90c88-0027-4dc7-a791-400e6ce97f20 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:29.842893 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d1dfd-63a8-4fec-862f-c1323d1481d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:29.843551 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d1dfd-63a8-4fec-862f-c1323d1481d1 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:29.843826 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d1dfd-63a8-4fec-862f-c1323d1481d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:29.844094 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d1dfd-63a8-4fec-862f-c1323d1481d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:29.883879 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964e2218-1441-4d5e-a9de-a72233668341 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:29.895045 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:29.912122 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d1dfd-63a8-4fec-862f-c1323d1481d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:29.913583 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d1dfd-63a8-4fec-862f-c1323d1481d1 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:29.918086 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7291a0d-63a8-4b62-b2e3-5e09e9d733ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:29.918707 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7291a0d-63a8-4b62-b2e3-5e09e9d733ee None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:29.919007 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7291a0d-63a8-4b62-b2e3-5e09e9d733ee None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:29.919278 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7291a0d-63a8-4b62-b2e3-5e09e9d733ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:29.929897 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7291a0d-63a8-4b62-b2e3-5e09e9d733ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:29.931422 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7291a0d-63a8-4b62-b2e3-5e09e9d733ee None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:29.946270 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-a7291a0d-63a8-4b62-b2e3-5e09e9d733ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:29.946270 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:29.946270 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:29.946270 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:29.946270 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:29.946270 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:29.946270 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:29.946270 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:29.946270 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:29.946270 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:29.946270 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:29.946270 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:29.946270 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:29.946270 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:29.946270 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:29.946270 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:29.947714 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:29.947714 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:29.947714 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:29.947714 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:29.947714 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:29.947714 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:29.947714 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:29.947714 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:29.947714 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:29.947714 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:29.947714 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:29.947714 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:29.947714 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:29.947714 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:29.947714 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:29.947714 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:29.947714 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:29.948971 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:29.948971 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:29.948971 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:29.948971 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 259/511] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:29.948971 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7291a0d-63a8-4b62-b2e3-5e09e9d733ee None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:29.996692 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:30.017183 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 253/512] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:54:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:54:30.018020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d1dfd-63a8-4fec-862f-c1323d1481d1 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:30.052504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7291a0d-63a8-4b62-b2e3-5e09e9d733ee None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:30.086477 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1012164f-7576-4fcf-b8da-054035f2c529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:30.087134 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012164f-7576-4fcf-b8da-054035f2c529 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:30.087411 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012164f-7576-4fcf-b8da-054035f2c529 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:30.087700 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012164f-7576-4fcf-b8da-054035f2c529 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:30.099143 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1012164f-7576-4fcf-b8da-054035f2c529 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:30.100540 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1012164f-7576-4fcf-b8da-054035f2c529 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:30.100980 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:30.114716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-1012164f-7576-4fcf-b8da-054035f2c529 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:30.114716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:30.114716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:30.114716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:30.114716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:30.114716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:30.114716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:30.114716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:30.114716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:30.114716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:30.114716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:30.114716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:30.114716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:30.114716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:30.114716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:30.114716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:30.115830 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:30.115830 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:30.115830 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:30.115830 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:30.115830 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:30.115830 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:30.115830 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:30.115830 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:30.115830 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:30.115830 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:30.115830 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:30.115830 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:30.115830 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:30.115830 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:30.115830 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:30.115830 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:30.115830 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:30.116894 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:30.116894 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:30.116894 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:30.117094 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d1dfd-63a8-4fec-862f-c1323d1481d1 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:30.120436 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 260/513] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:30.121207 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1012164f-7576-4fcf-b8da-054035f2c529 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:30.152555 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0933f8-b6ae-4d4d-8c10-beb431896a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:30.153154 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0933f8-b6ae-4d4d-8c10-beb431896a4b None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:30.153425 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0933f8-b6ae-4d4d-8c10-beb431896a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:30.153693 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0933f8-b6ae-4d4d-8c10-beb431896a4b None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:30.154321 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0933f8-b6ae-4d4d-8c10-beb431896a4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:30.154614 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0933f8-b6ae-4d4d-8c10-beb431896a4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:30.156465 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0933f8-b6ae-4d4d-8c10-beb431896a4b None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:30.168493 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 254/514] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:30.168825 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0933f8-b6ae-4d4d-8c10-beb431896a4b None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:30.221739 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:30.224877 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1012164f-7576-4fcf-b8da-054035f2c529 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:30.258291 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0abd5-74cd-4d51-8687-5dc043f5f099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:30.258945 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0abd5-74cd-4d51-8687-5dc043f5f099 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:30.259227 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0abd5-74cd-4d51-8687-5dc043f5f099 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:30.259498 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0abd5-74cd-4d51-8687-5dc043f5f099 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:30.267766 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d0abd5-74cd-4d51-8687-5dc043f5f099 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:30.267899 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0933f8-b6ae-4d4d-8c10-beb431896a4b None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:30.269251 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d0abd5-74cd-4d51-8687-5dc043f5f099 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:30.280130 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-81d0abd5-74cd-4d51-8687-5dc043f5f099 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 14:54:30.280130 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:30.280130 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:30.280130 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:30.280130 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:30.280130 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:30.280130 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:30.280130 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:30.280130 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:30.280130 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:30.280130 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:30.280130 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:30.280130 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:54:30.280130 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:54:30.280130 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:54:30.280130 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:54:30.286005 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:30.286005 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:30.286005 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:54:30.286005 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:30.286005 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:54:30.286005 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:30.286005 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:30.286005 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:30.286005 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:30.286005 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:30.286005 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:30.286005 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:30.286005 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:30.286005 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:30.286005 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:30.286005 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:30.286005 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:30.286005 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:30.286943 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:30.286943 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:30.286943 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:54:30.286943 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:54:30.286943 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:54:30.286943 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:54:30.286943 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:54:30.286943 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:54:30.286943 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 14:54:30.286943 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:30.286943 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 261/515] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:54:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:30.286943 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0abd5-74cd-4d51-8687-5dc043f5f099 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:30.309346 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02591b8-bbd4-4eaa-80aa-a7a4a049fa95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:30.310425 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02591b8-bbd4-4eaa-80aa-a7a4a049fa95 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:30.310912 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02591b8-bbd4-4eaa-80aa-a7a4a049fa95 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:30.311188 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02591b8-bbd4-4eaa-80aa-a7a4a049fa95 None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:30.312515 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02591b8-bbd4-4eaa-80aa-a7a4a049fa95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:30.312981 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02591b8-bbd4-4eaa-80aa-a7a4a049fa95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:30.314353 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02591b8-bbd4-4eaa-80aa-a7a4a049fa95 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:30.325966 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 255/516] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:30.326764 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02591b8-bbd4-4eaa-80aa-a7a4a049fa95 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:30.334598 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:30.380770 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0abd5-74cd-4d51-8687-5dc043f5f099 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:30.416415 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0516e-bd21-4573-aadb-61d5f572b684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:30.417043 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0516e-bd21-4573-aadb-61d5f572b684 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:30.417318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0516e-bd21-4573-aadb-61d5f572b684 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:30.417588 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0516e-bd21-4573-aadb-61d5f572b684 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:30.432232 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02591b8-bbd4-4eaa-80aa-a7a4a049fa95 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:30.433679 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d0516e-bd21-4573-aadb-61d5f572b684 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:30.435162 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d0516e-bd21-4573-aadb-61d5f572b684 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:30.455286 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-81d0516e-bd21-4573-aadb-61d5f572b684 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:30.455286 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:30.455286 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:30.455286 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:30.455286 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:30.455286 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:30.455286 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:30.455286 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:30.455286 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:30.455286 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:30.455286 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:30.455286 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:30.455286 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:30.455286 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:30.455286 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:30.455286 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:30.456305 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:30.456305 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:30.456305 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:30.456305 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:30.456305 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:30.456305 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:30.456305 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:30.456305 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:30.456305 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:30.456305 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:30.456305 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:30.456305 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:30.456305 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:30.456305 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:30.456305 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:30.456305 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:30.456305 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:30.457183 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:30.457183 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:30.457183 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:30.457183 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 262/517] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:30.457183 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0516e-bd21-4573-aadb-61d5f572b684 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:30.471397 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b897f0-f326-4569-bf91-6bd646087fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:30.472015 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b897f0-f326-4569-bf91-6bd646087fde None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:30.472295 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b897f0-f326-4569-bf91-6bd646087fde None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:30.472416 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:30.472570 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b897f0-f326-4569-bf91-6bd646087fde None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:30.473669 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b897f0-f326-4569-bf91-6bd646087fde None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:30.474063 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b897f0-f326-4569-bf91-6bd646087fde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:30.475537 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b897f0-f326-4569-bf91-6bd646087fde None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:30.486714 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 256/518] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:30.487609 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b897f0-f326-4569-bf91-6bd646087fde None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:30.550916 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:30.559817 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0516e-bd21-4573-aadb-61d5f572b684 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:30.588801 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b897f0-f326-4569-bf91-6bd646087fde None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:30.593484 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51508127-7042-424e-882a-247e02f1200f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:30.594094 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51508127-7042-424e-882a-247e02f1200f None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:30.594376 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51508127-7042-424e-882a-247e02f1200f None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:30.594648 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51508127-7042-424e-882a-247e02f1200f None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:30.595317 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51508127-7042-424e-882a-247e02f1200f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:30.596774 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51508127-7042-424e-882a-247e02f1200f None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:30.619366 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 263/519] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 14:54:30 2022] POST /identity/v3/users => generated 247 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:54:30.619973 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51508127-7042-424e-882a-247e02f1200f None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:30.628062 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac84fa-02b8-440c-b159-6f512e0842b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:30.628707 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac84fa-02b8-440c-b159-6f512e0842b4 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:30.628993 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac84fa-02b8-440c-b159-6f512e0842b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:30.629271 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac84fa-02b8-440c-b159-6f512e0842b4 None admin] PATH_INFO: `/v3/users` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:30.630024 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac84fa-02b8-440c-b159-6f512e0842b4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:30.630335 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac84fa-02b8-440c-b159-6f512e0842b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:30.631966 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac84fa-02b8-440c-b159-6f512e0842b4 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:30.654369 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 257/520] 10.222.0.33 () {56 vars in 1137 bytes} [Sun Sep 11 14:54:30 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:30.655914 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac84fa-02b8-440c-b159-6f512e0842b4 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:30.711612 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:30.754926 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51508127-7042-424e-882a-247e02f1200f None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:30.775683 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac84fa-02b8-440c-b159-6f512e0842b4 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:30.792130 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59561035-2318-4316-b813-227c4e33ba7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:30.792773 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59561035-2318-4316-b813-227c4e33ba7d None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:30.793053 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59561035-2318-4316-b813-227c4e33ba7d None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:30.793324 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59561035-2318-4316-b813-227c4e33ba7d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:30.801346 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59561035-2318-4316-b813-227c4e33ba7d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:30.803440 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59561035-2318-4316-b813-227c4e33ba7d None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:30.805140 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:30.817300 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-59561035-2318-4316-b813-227c4e33ba7d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:30.817300 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:30.817300 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:30.817300 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:30.817300 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:30.817300 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:30.817300 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:30.817300 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:30.817300 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:30.817300 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:30.817300 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:30.817300 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:30.817300 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:54:30.817300 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:54:30.817300 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:54:30.818280 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:54:30.818280 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:30.818280 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:30.818280 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:30.818280 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:30.818280 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:30.818280 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:30.818280 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:30.818280 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:30.818280 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:30.818280 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:30.818280 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:30.818280 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:30.818280 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:30.818280 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:30.818280 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:30.818280 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:30.819389 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:54:30.819389 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:54:30.819389 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:54:30.819389 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:54:30.819389 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:54:30.819389 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:54:30.819389 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:30.819389 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:30.820090 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 264/521] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:54:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:30.820883 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59561035-2318-4316-b813-227c4e33ba7d None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:30.822631 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172e8e6-e219-4197-9ff0-8d82fd6ff252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:30.823601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172e8e6-e219-4197-9ff0-8d82fd6ff252 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:30.824027 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172e8e6-e219-4197-9ff0-8d82fd6ff252 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:30.824515 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172e8e6-e219-4197-9ff0-8d82fd6ff252 None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:30.825172 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e172e8e6-e219-4197-9ff0-8d82fd6ff252 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:30.826237 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e172e8e6-e219-4197-9ff0-8d82fd6ff252 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:30.828274 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e172e8e6-e219-4197-9ff0-8d82fd6ff252 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:30.844266 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 258/522] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:30.845619 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172e8e6-e219-4197-9ff0-8d82fd6ff252 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:30.932259 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59561035-2318-4316-b813-227c4e33ba7d None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:30.956689 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172e8e6-e219-4197-9ff0-8d82fd6ff252 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:30.960367 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:30.969891 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf31e16-d344-49cb-937b-68dd336676a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:30.970862 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf31e16-d344-49cb-937b-68dd336676a4 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:30.971297 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf31e16-d344-49cb-937b-68dd336676a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:30.971580 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf31e16-d344-49cb-937b-68dd336676a4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:30.983680 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf31e16-d344-49cb-937b-68dd336676a4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:30.985295 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf31e16-d344-49cb-937b-68dd336676a4 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:30.997619 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46be1ab-5cf7-4068-8fa5-715683902b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:30.998241 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46be1ab-5cf7-4068-8fa5-715683902b56 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:30.998525 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46be1ab-5cf7-4068-8fa5-715683902b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:30.998833 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46be1ab-5cf7-4068-8fa5-715683902b56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:31.000812 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-1cf31e16-d344-49cb-937b-68dd336676a4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 14:54:31.000812 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:31.000812 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:31.000812 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:31.000812 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:31.000812 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:31.000812 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:31.000812 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:31.000812 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:31.000812 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:31.000812 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:31.000812 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:31.000812 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:54:31.000812 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:54:31.000812 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:54:31.000812 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:54:31.001664 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:31.001664 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:31.001664 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:54:31.001664 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:31.001664 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:54:31.001664 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:31.001664 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:31.001664 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:31.001664 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:31.001664 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:31.001664 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:31.001664 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:31.001664 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:31.001664 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:31.001664 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:31.001664 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:31.001664 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:31.001664 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:31.007571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:31.007571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:31.007571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:54:31.007571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:54:31.007571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:54:31.007571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:54:31.007571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:54:31.007571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:54:31.007571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 14:54:31.007571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:31.007571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 265/523] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 14:54:30 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:31.007571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf31e16-d344-49cb-937b-68dd336676a4 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:31.054349 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:31.077440 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46be1ab-5cf7-4068-8fa5-715683902b56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:31.079043 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46be1ab-5cf7-4068-8fa5-715683902b56 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:31.108737 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf31e16-d344-49cb-937b-68dd336676a4 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:31.146144 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15161d23-48b8-4d20-872e-5fee162de72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:31.146836 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15161d23-48b8-4d20-872e-5fee162de72d None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:31.147067 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15161d23-48b8-4d20-872e-5fee162de72d None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:31.147343 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15161d23-48b8-4d20-872e-5fee162de72d None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:31.149437 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15161d23-48b8-4d20-872e-5fee162de72d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:31.149437 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15161d23-48b8-4d20-872e-5fee162de72d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:31.149786 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15161d23-48b8-4d20-872e-5fee162de72d None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:31.170379 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 266/524] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:54:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:31.171842 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15161d23-48b8-4d20-872e-5fee162de72d None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:31.192823 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 259/525] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:54:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:54:31.193864 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46be1ab-5cf7-4068-8fa5-715683902b56 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:31.232209 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:31.279548 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:31.288294 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15161d23-48b8-4d20-872e-5fee162de72d None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:31.300301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46be1ab-5cf7-4068-8fa5-715683902b56 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:31.325541 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c27d53-e3b0-4963-ae9a-e45ad8a2c2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:31.326166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c27d53-e3b0-4963-ae9a-e45ad8a2c2ee None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:31.326447 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c27d53-e3b0-4963-ae9a-e45ad8a2c2ee None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:31.326722 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c27d53-e3b0-4963-ae9a-e45ad8a2c2ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:31.334240 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c27d53-e3b0-4963-ae9a-e45ad8a2c2ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:31.335741 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c27d53-e3b0-4963-ae9a-e45ad8a2c2ee None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:31.341522 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb5d2ca-9c92-4ac2-893f-8572ebf4e49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:31.342387 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb5d2ca-9c92-4ac2-893f-8572ebf4e49a None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:31.342997 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb5d2ca-9c92-4ac2-893f-8572ebf4e49a None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:31.343513 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb5d2ca-9c92-4ac2-893f-8572ebf4e49a None admin] PATH_INFO: `/v3/users` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:31.344432 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb5d2ca-9c92-4ac2-893f-8572ebf4e49a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:31.344933 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb5d2ca-9c92-4ac2-893f-8572ebf4e49a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:31.346670 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb5d2ca-9c92-4ac2-893f-8572ebf4e49a None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:31.347041 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-c4c27d53-e3b0-4963-ae9a-e45ad8a2c2ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:31.347041 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:31.347041 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:31.347041 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:31.347041 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:31.347041 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:31.347041 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:31.347041 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:31.347041 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:31.347041 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:31.347041 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:31.347041 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:31.347041 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:54:31.347041 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:54:31.347041 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:54:31.347863 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:54:31.347863 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:31.347863 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:31.347863 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:31.347863 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:31.347863 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:31.347863 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:31.347863 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:31.347863 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:31.347863 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:31.347863 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:31.347863 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:31.347863 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:31.347863 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:31.347863 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:31.347863 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:31.347863 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:31.348800 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:54:31.348800 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:54:31.348800 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:54:31.348800 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:54:31.348800 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:54:31.348800 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:54:31.348800 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:31.348800 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:31.348800 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 267/526] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:54:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:31.349261 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c27d53-e3b0-4963-ae9a-e45ad8a2c2ee None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:31.375721 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 260/527] 10.222.0.33 () {56 vars in 1139 bytes} [Sun Sep 11 14:54:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:31.375721 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb5d2ca-9c92-4ac2-893f-8572ebf4e49a None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:31.467007 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c27d53-e3b0-4963-ae9a-e45ad8a2c2ee None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:31.476526 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:31.481993 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb5d2ca-9c92-4ac2-893f-8572ebf4e49a None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:31.503556 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deea1fb4-c8d8-4656-be94-0857fc36fde0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:31.504360 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deea1fb4-c8d8-4656-be94-0857fc36fde0 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:31.504639 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deea1fb4-c8d8-4656-be94-0857fc36fde0 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:31.504915 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deea1fb4-c8d8-4656-be94-0857fc36fde0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:31.505641 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deea1fb4-c8d8-4656-be94-0857fc36fde0 None admin] RBAC: Adding query filter params (role.id=533fa341c4f64cd4b0a7ebb668f9106a, scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176, user.id=fc0ea93084b0482fb2dd6d802987942f) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:31.505952 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deea1fb4-c8d8-4656-be94-0857fc36fde0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:31.507490 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deea1fb4-c8d8-4656-be94-0857fc36fde0 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:31.513007 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 268/528] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:54:31 2022] GET /identity/v3/role_assignments?user.id=fc0ea93084b0482fb2dd6d802987942f&scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176&role.id=533fa341c4f64cd4b0a7ebb668f9106a => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:31.513788 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deea1fb4-c8d8-4656-be94-0857fc36fde0 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:31.541241 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117f6aa-5d12-46c3-a057-eb776a6128ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:31.541905 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117f6aa-5d12-46c3-a057-eb776a6128ca None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:31.542183 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117f6aa-5d12-46c3-a057-eb776a6128ca None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:31.542455 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117f6aa-5d12-46c3-a057-eb776a6128ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:31.555099 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8117f6aa-5d12-46c3-a057-eb776a6128ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:31.556658 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8117f6aa-5d12-46c3-a057-eb776a6128ca None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:31.567229 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:31.581055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-8117f6aa-5d12-46c3-a057-eb776a6128ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:31.581055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:31.581055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:31.581055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:31.581055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:31.581055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:31.581055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:31.581055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:31.581055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:31.581055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:31.581055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:31.581055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:31.581055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:31.581055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:31.581055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:31.581055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:31.582259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:31.582259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:31.582259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:31.582259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:31.582259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:31.582259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:31.582259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:31.582259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:31.582259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:31.582259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:31.582259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:31.582259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:31.582259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:31.582259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:31.582259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:31.582259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:31.582259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:31.583186 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:31.583186 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:31.583186 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:31.588387 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 261/529] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:31.589374 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117f6aa-5d12-46c3-a057-eb776a6128ca None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:31.669706 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deea1fb4-c8d8-4656-be94-0857fc36fde0 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:31.713798 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117f6aa-5d12-46c3-a057-eb776a6128ca None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:31.720537 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51496f-5f39-4060-af9d-45f670d2bcf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:31.721412 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51496f-5f39-4060-af9d-45f670d2bcf6 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:31.722364 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51496f-5f39-4060-af9d-45f670d2bcf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:31.722888 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51496f-5f39-4060-af9d-45f670d2bcf6 None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:31.723792 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc51496f-5f39-4060-af9d-45f670d2bcf6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:31.724610 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc51496f-5f39-4060-af9d-45f670d2bcf6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:31.726915 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc51496f-5f39-4060-af9d-45f670d2bcf6 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:31.745370 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 269/530] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:54:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:31.746401 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51496f-5f39-4060-af9d-45f670d2bcf6 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:31.758186 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964def75-e009-44e8-afe9-30992baa3a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:31.758915 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964def75-e009-44e8-afe9-30992baa3a8b None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:31.759266 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964def75-e009-44e8-afe9-30992baa3a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:31.759609 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964def75-e009-44e8-afe9-30992baa3a8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:31.788834 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:31.831729 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:31.847881 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964def75-e009-44e8-afe9-30992baa3a8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:31.850022 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964def75-e009-44e8-afe9-30992baa3a8b None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:31.864950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51496f-5f39-4060-af9d-45f670d2bcf6 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:31.904851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15422eb5-0a1f-4977-92c8-e4dedf870fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:31.906400 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15422eb5-0a1f-4977-92c8-e4dedf870fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:31.906752 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15422eb5-0a1f-4977-92c8-e4dedf870fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:31.907071 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15422eb5-0a1f-4977-92c8-e4dedf870fb8 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:31.907745 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15422eb5-0a1f-4977-92c8-e4dedf870fb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:31.908046 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15422eb5-0a1f-4977-92c8-e4dedf870fb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:31.909429 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15422eb5-0a1f-4977-92c8-e4dedf870fb8 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:31.929002 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 270/531] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:31.930327 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15422eb5-0a1f-4977-92c8-e4dedf870fb8 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:31.996600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 262/532] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:54:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:54:31.996600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964def75-e009-44e8-afe9-30992baa3a8b None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:32.062568 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:32.076650 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15422eb5-0a1f-4977-92c8-e4dedf870fb8 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:32.100456 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:32.112182 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964def75-e009-44e8-afe9-30992baa3a8b None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:32.126926 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d35b6-94b7-4a48-a6db-cd9c416344fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:32.127950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d35b6-94b7-4a48-a6db-cd9c416344fb None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:32.128819 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d35b6-94b7-4a48-a6db-cd9c416344fb None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:32.129266 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d35b6-94b7-4a48-a6db-cd9c416344fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:32.130390 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0d35b6-94b7-4a48-a6db-cd9c416344fb None admin] RBAC: Adding query filter params (role.id=533fa341c4f64cd4b0a7ebb668f9106a, scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176, user.id=cd9c8b53871e415facea40c6694f4171) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:32.131128 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0d35b6-94b7-4a48-a6db-cd9c416344fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:32.133089 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0d35b6-94b7-4a48-a6db-cd9c416344fb None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:32.141173 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 271/533] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:54:31 2022] GET /identity/v3/role_assignments?user.id=cd9c8b53871e415facea40c6694f4171&scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176&role.id=533fa341c4f64cd4b0a7ebb668f9106a => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:32.142117 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d35b6-94b7-4a48-a6db-cd9c416344fb None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:32.176323 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0309f7-3146-43c5-978b-6b24cc4248b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:32.177233 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0309f7-3146-43c5-978b-6b24cc4248b3 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:32.177616 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0309f7-3146-43c5-978b-6b24cc4248b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:32.177990 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0309f7-3146-43c5-978b-6b24cc4248b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:32.196013 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0309f7-3146-43c5-978b-6b24cc4248b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:32.197792 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0309f7-3146-43c5-978b-6b24cc4248b3 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:32.222307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-9f0309f7-3146-43c5-978b-6b24cc4248b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:32.222307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:32.222307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:32.222307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:32.222307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:32.222307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:32.222307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:32.222307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:32.222307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:32.222307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:32.222307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:32.222307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:32.222307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:54:32.222307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:54:32.222307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:54:32.223288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:54:32.223288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:32.223288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:32.223288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:32.223288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:32.223288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:32.223288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:32.223288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:32.223288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:32.223288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:32.223288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:32.223288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:32.223288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:32.223288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:32.223288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:32.223288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:32.223288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:32.224145 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:54:32.224145 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:54:32.224145 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:54:32.224145 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:54:32.224145 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:54:32.224145 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:54:32.224145 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:32.224145 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:32.230921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 263/534] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:54:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:32.235429 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0309f7-3146-43c5-978b-6b24cc4248b3 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:32.310503 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d35b6-94b7-4a48-a6db-cd9c416344fb None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:32.377117 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0309f7-3146-43c5-978b-6b24cc4248b3 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:32.379317 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2d38f0-a09e-4934-a469-dfd55824631b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:32.379984 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d38f0-a09e-4934-a469-dfd55824631b None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:32.380362 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d38f0-a09e-4934-a469-dfd55824631b None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:32.381379 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d38f0-a09e-4934-a469-dfd55824631b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:32.400890 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2d38f0-a09e-4934-a469-dfd55824631b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:32.402632 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2d38f0-a09e-4934-a469-dfd55824631b None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:32.430064 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772dea45-cab8-4543-9d76-0fa618aafeae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:32.430693 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772dea45-cab8-4543-9d76-0fa618aafeae None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:32.431004 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772dea45-cab8-4543-9d76-0fa618aafeae None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:32.431275 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772dea45-cab8-4543-9d76-0fa618aafeae None admin] PATH_INFO: `/v3/projects` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:32.431982 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772dea45-cab8-4543-9d76-0fa618aafeae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:32.432283 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772dea45-cab8-4543-9d76-0fa618aafeae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:32.434018 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772dea45-cab8-4543-9d76-0fa618aafeae None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:32.441508 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-aa2d38f0-a09e-4934-a469-dfd55824631b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:32.441508 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:32.441508 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:32.441508 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:32.441508 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:32.441508 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:32.441508 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:32.441508 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:32.441508 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:32.441508 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:32.441508 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:32.441508 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:32.441508 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:32.441508 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:32.441508 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:32.441508 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:32.442455 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:32.442455 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:32.442455 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:32.442455 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:32.442455 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:32.442455 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:32.442455 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:32.442455 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:32.442455 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:32.442455 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:32.442455 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:32.442455 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:32.442455 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:32.442455 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:32.442455 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:32.442455 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:32.442455 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:32.443331 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:32.443331 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:32.443331 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:32.443331 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 272/535] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:32.448491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2d38f0-a09e-4934-a469-dfd55824631b None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:32.461137 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:32.469754 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 264/536] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:54:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:54:32.475589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772dea45-cab8-4543-9d76-0fa618aafeae None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:32.508936 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:32.605827 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772dea45-cab8-4543-9d76-0fa618aafeae None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:32.643842 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf73aa6-a0fa-4cff-98df-64637272f440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:32.644584 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf73aa6-a0fa-4cff-98df-64637272f440 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:32.644870 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf73aa6-a0fa-4cff-98df-64637272f440 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:32.645145 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf73aa6-a0fa-4cff-98df-64637272f440 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:32.645980 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf73aa6-a0fa-4cff-98df-64637272f440 None admin] RBAC: Adding query filter params (role.id=533fa341c4f64cd4b0a7ebb668f9106a, scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176, user.id=c793eafcdba249a5af3ab948e9774367) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:32.646290 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf73aa6-a0fa-4cff-98df-64637272f440 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:32.650827 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf73aa6-a0fa-4cff-98df-64637272f440 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:32.654973 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 265/537] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:54:32 2022] GET /identity/v3/role_assignments?user.id=c793eafcdba249a5af3ab948e9774367&scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176&role.id=533fa341c4f64cd4b0a7ebb668f9106a => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:54:32.676989 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2d38f0-a09e-4934-a469-dfd55824631b None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:32.722040 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab5093-e1c8-41f5-8d26-260e51292b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:32.722941 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab5093-e1c8-41f5-8d26-260e51292b5d None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:32.723369 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab5093-e1c8-41f5-8d26-260e51292b5d None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:32.723791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab5093-e1c8-41f5-8d26-260e51292b5d None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:32.724620 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ab5093-e1c8-41f5-8d26-260e51292b5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:32.725171 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ab5093-e1c8-41f5-8d26-260e51292b5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:32.727378 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ab5093-e1c8-41f5-8d26-260e51292b5d None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:32.748815 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 273/538] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:54:32.754174 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf73aa6-a0fa-4cff-98df-64637272f440 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:32.891543 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:32.905550 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088cb991-f36f-4f82-8ab7-19d752950285 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:32.905849 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088cb991-f36f-4f82-8ab7-19d752950285 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:32.906098 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088cb991-f36f-4f82-8ab7-19d752950285 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:32.907370 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf73aa6-a0fa-4cff-98df-64637272f440 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:32.908354 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 274/539] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:32.914469 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410a782-ae41-43ec-ac91-08ae30778b84 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:32.914745 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410a782-ae41-43ec-ac91-08ae30778b84 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:32.915045 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410a782-ae41-43ec-ac91-08ae30778b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:32.935371 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-6410a782-ae41-43ec-ac91-08ae30778b84 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:32.937088 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6410a782-ae41-43ec-ac91-08ae30778b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:32.992131 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913dc765-4638-4532-980e-1302e0383289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:32.992131 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913dc765-4638-4532-980e-1302e0383289 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:32.992390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913dc765-4638-4532-980e-1302e0383289 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:32.993116 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913dc765-4638-4532-980e-1302e0383289 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:33.005006 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913dc765-4638-4532-980e-1302e0383289 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:33.006726 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913dc765-4638-4532-980e-1302e0383289 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:33.029796 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-913dc765-4638-4532-980e-1302e0383289 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:33.029796 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:33.029796 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:33.029796 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:33.029796 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:33.029796 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:33.029796 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:33.029796 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:33.029796 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:33.029796 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:33.029796 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:33.029796 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:33.029796 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:54:33.029796 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:54:33.029796 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:54:33.030777 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:54:33.030777 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:33.030777 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:33.030777 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:33.030777 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:33.030777 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:33.030777 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:33.030777 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:33.030777 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:33.030777 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:33.030777 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:33.030777 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:33.030777 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:33.030777 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:33.030777 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:33.030777 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:33.030777 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:33.031671 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:54:33.031671 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:54:33.031671 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:54:33.031671 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:54:33.031671 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:54:33.031671 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:54:33.031671 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:33.031671 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:33.034467 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 266/540] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:54:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:33.040612 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:33.041702 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913dc765-4638-4532-980e-1302e0383289 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:33.048710 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 275/541] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:54:33.211753 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913dc765-4638-4532-980e-1302e0383289 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:33.266161 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f273e3-74fe-47ec-a0cf-eca73619c778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:33.266784 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f273e3-74fe-47ec-a0cf-eca73619c778 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:33.267098 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f273e3-74fe-47ec-a0cf-eca73619c778 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:33.267372 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f273e3-74fe-47ec-a0cf-eca73619c778 None admin] PATH_INFO: `/v3/projects` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:33.268070 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f273e3-74fe-47ec-a0cf-eca73619c778 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:33.268380 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f273e3-74fe-47ec-a0cf-eca73619c778 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:33.269832 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f273e3-74fe-47ec-a0cf-eca73619c778 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:33.275936 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:33.291990 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 267/542] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:54:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:33.297553 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410a782-ae41-43ec-ac91-08ae30778b84 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:33.442923 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410a782-ae41-43ec-ac91-08ae30778b84 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:33.482270 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fabc046-ec4d-407d-90d9-ca1322f1923c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:33.482931 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fabc046-ec4d-407d-90d9-ca1322f1923c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:33.483206 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fabc046-ec4d-407d-90d9-ca1322f1923c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:33.484028 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fabc046-ec4d-407d-90d9-ca1322f1923c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:33.484556 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f273e3-74fe-47ec-a0cf-eca73619c778 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:33.484757 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fabc046-ec4d-407d-90d9-ca1322f1923c None admin] RBAC: Adding query filter params (role.id=533fa341c4f64cd4b0a7ebb668f9106a, scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176, user.id=df6eb236e80b4395a7aa77020b2f3a4f) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:33.485059 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fabc046-ec4d-407d-90d9-ca1322f1923c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:33.486635 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fabc046-ec4d-407d-90d9-ca1322f1923c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:33.492938 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 276/543] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:54:33 2022] GET /identity/v3/role_assignments?user.id=df6eb236e80b4395a7aa77020b2f3a4f&scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176&role.id=533fa341c4f64cd4b0a7ebb668f9106a => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:54:33.667828 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f273e3-74fe-47ec-a0cf-eca73619c778 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:33.721950 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67bea42-c304-4343-ba43-ef5f6c72ad16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:33.722623 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67bea42-c304-4343-ba43-ef5f6c72ad16 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:33.722932 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67bea42-c304-4343-ba43-ef5f6c72ad16 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:33.723211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67bea42-c304-4343-ba43-ef5f6c72ad16 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:33.745970 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67bea42-c304-4343-ba43-ef5f6c72ad16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:33.751023 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-b67bea42-c304-4343-ba43-ef5f6c72ad16 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:54:33.751023 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:33.751023 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:33.751023 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:33.751023 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:33.751023 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:33.751023 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:33.751023 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:33.751023 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:33.751023 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:33.751023 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:33.751023 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:33.751023 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:33.751023 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:33.751023 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:33.751023 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:33.751907 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:33.751907 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:33.751907 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:33.751907 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:33.751907 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:33.751907 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:33.751907 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:33.751907 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:33.751907 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:33.751907 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:33.751907 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:33.751907 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:33.751907 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:33.751907 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:33.751907 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:33.751907 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:33.752695 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:33.752695 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:33.752695 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:33.752695 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:33.752695 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:33.752695 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:33.752695 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:33.752695 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:33.752695 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:33.752695 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:33.752695 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:54:33.752695 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:33.752695 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 268/544] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 14:54:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:33.758182 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fabc046-ec4d-407d-90d9-ca1322f1923c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:33.853979 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:33.918515 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fabc046-ec4d-407d-90d9-ca1322f1923c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:33.964508 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e845b-5d8c-422c-9233-890a9f380b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:33.965139 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e845b-5d8c-422c-9233-890a9f380b7e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:33.965417 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e845b-5d8c-422c-9233-890a9f380b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:33.965686 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e845b-5d8c-422c-9233-890a9f380b7e None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:33.966362 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63e845b-5d8c-422c-9233-890a9f380b7e None admin] RBAC: Adding query filter params (name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:33.966661 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63e845b-5d8c-422c-9233-890a9f380b7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:33.968756 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63e845b-5d8c-422c-9233-890a9f380b7e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:33.978595 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 277/545] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 14:54:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:33.984588 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67bea42-c304-4343-ba43-ef5f6c72ad16 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:34.091024 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:34.113269 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3962d-818f-4d48-a34f-f4820c308212 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:34.113811 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3962d-818f-4d48-a34f-f4820c308212 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:34.114242 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3962d-818f-4d48-a34f-f4820c308212 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:34.116006 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 278/546] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:34.120675 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce4dd8-ec71-4c52-9768-d4d1817b5f76 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:34.121120 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce4dd8-ec71-4c52-9768-d4d1817b5f76 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:34.121533 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce4dd8-ec71-4c52-9768-d4d1817b5f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:34.145472 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67bea42-c304-4343-ba43-ef5f6c72ad16 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:34.159268 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-55ce4dd8-ec71-4c52-9768-d4d1817b5f76 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:34.160752 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-55ce4dd8-ec71-4c52-9768-d4d1817b5f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:34.184610 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13248f3-a0eb-43ea-a8d9-9a3bcfe442e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:34.185282 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13248f3-a0eb-43ea-a8d9-9a3bcfe442e2 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:34.185638 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13248f3-a0eb-43ea-a8d9-9a3bcfe442e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:34.186074 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13248f3-a0eb-43ea-a8d9-9a3bcfe442e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:34.197151 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13248f3-a0eb-43ea-a8d9-9a3bcfe442e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:34.198929 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13248f3-a0eb-43ea-a8d9-9a3bcfe442e2 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:34.215262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-a13248f3-a0eb-43ea-a8d9-9a3bcfe442e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:34.215262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:34.215262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:34.215262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:34.215262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:34.215262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:34.215262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:34.215262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:34.215262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:34.215262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:34.215262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:34.215262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:34.215262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:34.215262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:34.215262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:34.215262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:34.216835 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:34.216835 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:34.216835 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:34.216835 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:34.216835 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:34.216835 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:34.216835 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:34.216835 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:34.216835 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:34.216835 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:34.216835 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:34.216835 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:34.216835 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:34.216835 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:34.216835 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:34.216835 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:34.216835 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:34.217950 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:34.217950 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:34.217950 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:34.220253 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 269/547] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:34.225274 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13248f3-a0eb-43ea-a8d9-9a3bcfe442e2 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:34.314017 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 279/548] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:54:34.374329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13248f3-a0eb-43ea-a8d9-9a3bcfe442e2 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:34.383887 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:34.410224 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89c892-ab01-41b7-959c-37ce705d5e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:34.411161 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89c892-ab01-41b7-959c-37ce705d5e10 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:34.411677 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89c892-ab01-41b7-959c-37ce705d5e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:34.412111 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89c892-ab01-41b7-959c-37ce705d5e10 None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:34.412894 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89c892-ab01-41b7-959c-37ce705d5e10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:34.413344 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89c892-ab01-41b7-959c-37ce705d5e10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:34.414907 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89c892-ab01-41b7-959c-37ce705d5e10 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:34.428426 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 270/549] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:34.433900 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ce4dd8-ec71-4c52-9768-d4d1817b5f76 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:34.493725 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:34.548928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ce4dd8-ec71-4c52-9768-d4d1817b5f76 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:34.600594 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a614050-ab54-4b5a-94c0-e8f505cf8db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:34.601485 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a614050-ab54-4b5a-94c0-e8f505cf8db8 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:34.601912 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a614050-ab54-4b5a-94c0-e8f505cf8db8 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:34.602361 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a614050-ab54-4b5a-94c0-e8f505cf8db8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:34.615862 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a614050-ab54-4b5a-94c0-e8f505cf8db8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:34.617501 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a614050-ab54-4b5a-94c0-e8f505cf8db8 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:34.636538 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-2a614050-ab54-4b5a-94c0-e8f505cf8db8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 14:54:34.636538 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:34.636538 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:34.636538 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:34.636538 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:34.636538 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:34.636538 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:34.636538 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:34.636538 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:34.636538 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:34.636538 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:34.636538 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:34.636538 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:54:34.636538 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:54:34.636538 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:54:34.636538 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:54:34.638253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:34.638253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:34.638253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:54:34.638253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:34.638253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:54:34.638253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:34.638253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:34.638253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:34.638253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:34.638253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:34.638253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:34.638253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:34.638253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:34.638253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:34.638253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:34.638253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:34.638253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:34.638253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:34.639530 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:34.639530 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:34.639530 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:54:34.639530 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:54:34.639530 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:54:34.639530 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:54:34.639530 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:54:34.639530 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:54:34.639530 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 14:54:34.639530 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:34.641358 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 280/550] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:54:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:54:34.647087 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89c892-ab01-41b7-959c-37ce705d5e10 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:34.684872 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a614050-ab54-4b5a-94c0-e8f505cf8db8 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:34.775619 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89c892-ab01-41b7-959c-37ce705d5e10 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:34.813752 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147952d7-c929-4a5f-be2f-be38c89abef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:34.814367 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147952d7-c929-4a5f-be2f-be38c89abef4 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:34.814646 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147952d7-c929-4a5f-be2f-be38c89abef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:34.814961 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147952d7-c929-4a5f-be2f-be38c89abef4 None admin] PATH_INFO: `/v3/users` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:34.815752 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147952d7-c929-4a5f-be2f-be38c89abef4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:34.816063 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147952d7-c929-4a5f-be2f-be38c89abef4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:34.817518 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147952d7-c929-4a5f-be2f-be38c89abef4 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:34.823406 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a614050-ab54-4b5a-94c0-e8f505cf8db8 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:34.839429 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:34.843446 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 271/551] 10.222.0.33 () {56 vars in 1135 bytes} [Sun Sep 11 14:54:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:34.845958 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9969acc-d049-490e-be08-5b24f4d57157 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:34.846445 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9969acc-d049-490e-be08-5b24f4d57157 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:34.846954 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9969acc-d049-490e-be08-5b24f4d57157 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:34.848450 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 272/552] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:34.853415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9969acc-d049-490e-be08-5b24f4d57157 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:34.865953 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c9fe8b-45e0-4b98-8107-e2215776e9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:34.866587 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9fe8b-45e0-4b98-8107-e2215776e9e3 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:34.866908 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9fe8b-45e0-4b98-8107-e2215776e9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:34.867188 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9fe8b-45e0-4b98-8107-e2215776e9e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:34.879426 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c9fe8b-45e0-4b98-8107-e2215776e9e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:34.885354 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-70c9fe8b-45e0-4b98-8107-e2215776e9e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:54:34.885354 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:34.885354 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:34.885354 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:34.885354 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:34.885354 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:34.885354 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:34.885354 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:34.885354 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:34.885354 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:34.885354 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:34.885354 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:34.885354 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:34.885354 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:34.885354 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:34.885354 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:34.886355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:34.886355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:34.886355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:34.886355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:34.886355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:34.886355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:34.886355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:34.886355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:34.886355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:34.886355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:34.886355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:34.886355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:34.886355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:34.886355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:34.886355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:34.886355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:34.887819 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:34.887819 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:34.887819 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:34.887819 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:34.887819 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:34.887819 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:34.887819 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:34.887819 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:34.887819 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:34.887819 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:34.887819 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:54:34.887819 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:34.887819 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 281/553] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 14:54:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:54:34.888532 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f615368-9002-4368-b876-5b4e74908ac4 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:34.888532 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f615368-9002-4368-b876-5b4e74908ac4 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:34.888856 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f615368-9002-4368-b876-5b4e74908ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:34.892333 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:34.911044 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-9f615368-9002-4368-b876-5b4e74908ac4 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:34.912426 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-9f615368-9002-4368-b876-5b4e74908ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:34.985694 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9969acc-d049-490e-be08-5b24f4d57157 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:34.998943 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 282/554] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:35.000112 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f615368-9002-4368-b876-5b4e74908ac4 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:35.048706 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe2b1b-498b-4c62-b5a9-9db7cedd7542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:35.049730 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe2b1b-498b-4c62-b5a9-9db7cedd7542 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:35.050269 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe2b1b-498b-4c62-b5a9-9db7cedd7542 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:35.050744 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe2b1b-498b-4c62-b5a9-9db7cedd7542 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:35.164231 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f615368-9002-4368-b876-5b4e74908ac4 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:35.170531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:35.195812 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafe2b1b-498b-4c62-b5a9-9db7cedd7542 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:35.198518 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafe2b1b-498b-4c62-b5a9-9db7cedd7542 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:35.232433 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:35.255592 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af6a302-2d07-4676-a8cd-4f53013e49f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:35.256519 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6a302-2d07-4676-a8cd-4f53013e49f3 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:35.256519 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6a302-2d07-4676-a8cd-4f53013e49f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:35.256777 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6a302-2d07-4676-a8cd-4f53013e49f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:35.257456 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af6a302-2d07-4676-a8cd-4f53013e49f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:35.257759 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af6a302-2d07-4676-a8cd-4f53013e49f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:35.259233 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af6a302-2d07-4676-a8cd-4f53013e49f3 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:35.278465 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 283/555] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 14:54:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:35.284212 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af6a302-2d07-4676-a8cd-4f53013e49f3 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:35.321347 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 273/556] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:54:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:54:35.327340 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe2b1b-498b-4c62-b5a9-9db7cedd7542 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:35.428569 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af6a302-2d07-4676-a8cd-4f53013e49f3 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:35.448420 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe2b1b-498b-4c62-b5a9-9db7cedd7542 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:35.466020 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352839f-66c4-438b-a586-58dbbf8e3383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:35.466652 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352839f-66c4-438b-a586-58dbbf8e3383 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:35.466970 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352839f-66c4-438b-a586-58dbbf8e3383 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:35.467241 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352839f-66c4-438b-a586-58dbbf8e3383 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:35.477497 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:35.478682 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0352839f-66c4-438b-a586-58dbbf8e3383 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:35.480587 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0352839f-66c4-438b-a586-58dbbf8e3383 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:35.488267 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adea39e-3323-48b9-a11c-33cf04fd8e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:35.488900 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adea39e-3323-48b9-a11c-33cf04fd8e55 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:35.489178 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adea39e-3323-48b9-a11c-33cf04fd8e55 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:35.489841 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adea39e-3323-48b9-a11c-33cf04fd8e55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:35.495724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-0352839f-66c4-438b-a586-58dbbf8e3383 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:35.495724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:35.495724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:35.495724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:35.495724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:35.495724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:35.495724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:35.495724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:35.495724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:35.495724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:35.495724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:35.495724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:35.495724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:35.495724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:35.495724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:35.495724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:35.496665 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:35.496665 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:35.496665 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:35.496665 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:35.496665 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:35.496665 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:35.496665 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:35.496665 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:35.496665 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:35.496665 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:35.496665 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:35.496665 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:35.496665 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:35.496665 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:35.496665 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:35.496665 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:35.496665 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:35.497551 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:35.497551 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:35.497551 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:35.497551 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 284/557] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:54:35.497771 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352839f-66c4-438b-a586-58dbbf8e3383 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:35.505708 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adea39e-3323-48b9-a11c-33cf04fd8e55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:35.507202 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adea39e-3323-48b9-a11c-33cf04fd8e55 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:35.518717 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:35.523071 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-9adea39e-3323-48b9-a11c-33cf04fd8e55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:35.523071 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:35.523071 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:35.523071 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:35.523071 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:35.523071 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:35.523071 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:35.523071 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:35.523071 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:35.523071 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:35.523071 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:35.523071 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:35.523071 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:35.523071 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:35.523071 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:35.523071 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:35.524457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:35.524457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:35.524457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:35.524457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:35.524457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:35.524457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:35.524457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:35.524457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:35.524457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:35.524457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:35.524457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:35.524457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:35.524457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:35.524457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:35.524457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:35.524457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:35.524457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:35.525357 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:35.525357 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:35.525357 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:35.525357 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 274/558] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:35.526114 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36beb8-7f7a-4b0f-bcfa-5617f41e16b2 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:35.526114 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36beb8-7f7a-4b0f-bcfa-5617f41e16b2 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:35.526361 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36beb8-7f7a-4b0f-bcfa-5617f41e16b2 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:35.527709 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 275/559] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:35.528301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36beb8-7f7a-4b0f-bcfa-5617f41e16b2 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:35.619569 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352839f-66c4-438b-a586-58dbbf8e3383 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:35.640791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36beb8-7f7a-4b0f-bcfa-5617f41e16b2 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:35.658266 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257a3e4-c830-46f1-8055-74f93d3e457f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:35.658930 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257a3e4-c830-46f1-8055-74f93d3e457f None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:35.659207 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257a3e4-c830-46f1-8055-74f93d3e457f None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:35.659478 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257a3e4-c830-46f1-8055-74f93d3e457f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:35.669886 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5257a3e4-c830-46f1-8055-74f93d3e457f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:35.675412 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-5257a3e4-c830-46f1-8055-74f93d3e457f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:54:35.675412 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:35.675412 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:35.675412 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:35.675412 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:35.675412 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:35.675412 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:35.675412 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:35.675412 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:35.675412 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:35.675412 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:35.675412 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:35.675412 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:35.675412 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:35.675412 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:35.675412 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:35.676368 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:35.676368 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:35.676368 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:35.676368 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:35.676368 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:35.676368 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:35.676368 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:35.676368 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:35.676368 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:35.676368 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:35.676368 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:35.676368 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:35.676368 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:35.676368 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:35.676368 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:35.676368 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:35.677320 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:35.677320 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:35.677320 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:35.677320 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:35.677320 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:35.677320 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:35.677320 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:35.677320 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:35.677320 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:35.677320 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:35.677320 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:54:35.677320 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:35.677320 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 285/560] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 14:54:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:35.677320 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257a3e4-c830-46f1-8055-74f93d3e457f None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:35.683716 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69615cdc-024a-4c0f-9644-e6d2b6911d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:35.684280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69615cdc-024a-4c0f-9644-e6d2b6911d46 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:35.684565 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69615cdc-024a-4c0f-9644-e6d2b6911d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:35.684840 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69615cdc-024a-4c0f-9644-e6d2b6911d46 None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:35.685494 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69615cdc-024a-4c0f-9644-e6d2b6911d46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:35.685801 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69615cdc-024a-4c0f-9644-e6d2b6911d46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:35.687521 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69615cdc-024a-4c0f-9644-e6d2b6911d46 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:35.700540 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 276/561] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:35.701582 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ce570-d316-4c03-a6a3-f9e0fba441f8 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:35.701867 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ce570-d316-4c03-a6a3-f9e0fba441f8 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:35.702713 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ce570-d316-4c03-a6a3-f9e0fba441f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:35.723642 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-841ce570-d316-4c03-a6a3-f9e0fba441f8 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:35.724844 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-841ce570-d316-4c03-a6a3-f9e0fba441f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:35.757644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:35.763318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:35.785989 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257a3e4-c830-46f1-8055-74f93d3e457f None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:35.810841 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 277/562] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:35.810841 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ce570-d316-4c03-a6a3-f9e0fba441f8 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:35.833973 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0602e668-599d-4ca7-a437-e29f17659913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:35.834588 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602e668-599d-4ca7-a437-e29f17659913 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:35.834895 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602e668-599d-4ca7-a437-e29f17659913 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:35.835172 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602e668-599d-4ca7-a437-e29f17659913 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:35.835815 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0602e668-599d-4ca7-a437-e29f17659913 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:35.836116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0602e668-599d-4ca7-a437-e29f17659913 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:35.837635 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0602e668-599d-4ca7-a437-e29f17659913 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:35.850617 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 286/563] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:35.851748 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0602e668-599d-4ca7-a437-e29f17659913 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:35.948431 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ce570-d316-4c03-a6a3-f9e0fba441f8 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:35.966887 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0602e668-599d-4ca7-a437-e29f17659913 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:36.000514 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a0a9a2-3f26-4022-9d6d-8905d7cb1657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:36.001779 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0a9a2-3f26-4022-9d6d-8905d7cb1657 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:36.002489 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0a9a2-3f26-4022-9d6d-8905d7cb1657 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:36.002961 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0a9a2-3f26-4022-9d6d-8905d7cb1657 None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:36.004030 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a0a9a2-3f26-4022-9d6d-8905d7cb1657 None admin] RBAC: Adding query filter params (name=service) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:36.004888 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a0a9a2-3f26-4022-9d6d-8905d7cb1657 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:36.006683 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a0a9a2-3f26-4022-9d6d-8905d7cb1657 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:36.012998 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2476d4b-2b1a-4719-ae7f-f6cd2702471e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:36.013888 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2476d4b-2b1a-4719-ae7f-f6cd2702471e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:36.014244 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2476d4b-2b1a-4719-ae7f-f6cd2702471e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:36.015288 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2476d4b-2b1a-4719-ae7f-f6cd2702471e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:36.018224 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 278/564] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 14:54:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:36.019656 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a0a9a2-3f26-4022-9d6d-8905d7cb1657 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:36.027321 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2476d4b-2b1a-4719-ae7f-f6cd2702471e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:36.028273 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2476d4b-2b1a-4719-ae7f-f6cd2702471e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:36.030454 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:36.045629 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-c2476d4b-2b1a-4719-ae7f-f6cd2702471e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 14:54:36.045629 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:36.045629 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:36.045629 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:36.045629 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:36.045629 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:36.045629 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:36.045629 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:36.045629 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:36.045629 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:36.045629 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:36.045629 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:36.045629 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:54:36.045629 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:54:36.045629 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:54:36.045629 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:54:36.046754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:36.046754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:36.046754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:54:36.046754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:36.046754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:54:36.046754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:36.046754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:36.046754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:36.046754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:36.046754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:36.046754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:36.046754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:36.046754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:36.046754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:36.046754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:36.046754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:36.046754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:36.046754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:36.047755 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:36.047755 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:36.047755 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:54:36.047755 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:54:36.047755 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:54:36.047755 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:54:36.047755 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:54:36.047755 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:54:36.047755 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 14:54:36.047755 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:36.050728 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 287/565] 10.222.0.33 () {56 vars in 1093 bytes} [Sun Sep 11 14:54:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:36.051930 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2476d4b-2b1a-4719-ae7f-f6cd2702471e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:36.060289 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:36.133930 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a0a9a2-3f26-4022-9d6d-8905d7cb1657 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:36.156383 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2476d4b-2b1a-4719-ae7f-f6cd2702471e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:36.167420 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc8e7c7-9883-44a6-853f-ca0280ded4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:36.168052 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8e7c7-9883-44a6-853f-ca0280ded4a3 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:36.168330 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8e7c7-9883-44a6-853f-ca0280ded4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:36.168605 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8e7c7-9883-44a6-853f-ca0280ded4a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:36.178015 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc8e7c7-9883-44a6-853f-ca0280ded4a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:36.179525 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc8e7c7-9883-44a6-853f-ca0280ded4a3 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:36.190201 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80279de-9daf-4fdc-a09a-10e8ccaa473a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:36.190827 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80279de-9daf-4fdc-a09a-10e8ccaa473a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:36.190982 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-ffc8e7c7-9883-44a6-853f-ca0280ded4a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:36.190982 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:36.190982 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:36.190982 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:36.190982 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:36.190982 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:36.190982 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:36.190982 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:36.190982 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:36.190982 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:36.190982 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:36.190982 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:36.190982 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:54:36.190982 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:54:36.190982 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:54:36.191973 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:54:36.191973 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:36.191973 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:36.191973 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:36.191973 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:36.191973 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:36.191973 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:36.191973 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:36.191973 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:36.191973 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:36.191973 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:36.191973 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:36.191973 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:36.191973 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:36.191973 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:36.191973 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:36.191973 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:36.197824 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:54:36.197824 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:54:36.197824 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:54:36.197824 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:54:36.197824 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:54:36.197824 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:54:36.197824 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:36.197824 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:36.198299 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80279de-9daf-4fdc-a09a-10e8ccaa473a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:36.198299 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80279de-9daf-4fdc-a09a-10e8ccaa473a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:36.198483 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 279/566] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:54:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:36.198483 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8512fd-9049-489b-8631-42b9724b6005 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:36.198483 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8512fd-9049-489b-8631-42b9724b6005 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:36.198483 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8512fd-9049-489b-8631-42b9724b6005 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:36.198483 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 280/567] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:36.198483 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8512fd-9049-489b-8631-42b9724b6005 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:36.208483 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80279de-9daf-4fdc-a09a-10e8ccaa473a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:36.209960 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80279de-9daf-4fdc-a09a-10e8ccaa473a None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:36.224416 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-f80279de-9daf-4fdc-a09a-10e8ccaa473a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:36.224416 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:36.224416 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:36.224416 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:36.224416 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:36.224416 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:36.224416 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:36.224416 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:36.224416 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:36.224416 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:36.224416 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:36.224416 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:36.224416 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:36.224416 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:36.224416 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:36.224416 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:36.225491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:36.225491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:36.225491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:36.225491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:36.225491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:36.225491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:36.225491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:36.225491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:36.225491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:36.225491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:36.225491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:36.225491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:36.225491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:36.225491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:36.225491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:36.225491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:36.225491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:36.226538 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:36.226538 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:36.226538 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:36.226538 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 288/568] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:36.226538 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80279de-9daf-4fdc-a09a-10e8ccaa473a None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:36.276684 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:36.293223 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:36.305749 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8512fd-9049-489b-8631-42b9724b6005 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:36.327773 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80279de-9daf-4fdc-a09a-10e8ccaa473a None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:36.340436 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b90a850-7e37-4420-a148-55c5457f41d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:36.341045 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90a850-7e37-4420-a148-55c5457f41d4 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:36.341325 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90a850-7e37-4420-a148-55c5457f41d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:36.341599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90a850-7e37-4420-a148-55c5457f41d4 None admin] PATH_INFO: `/v3/users` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:36.342319 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b90a850-7e37-4420-a148-55c5457f41d4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:36.342684 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b90a850-7e37-4420-a148-55c5457f41d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:36.344153 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b90a850-7e37-4420-a148-55c5457f41d4 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:36.360632 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ce6d0-7010-494f-a7fd-333be1ab2fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:36.360972 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 281/569] 10.222.0.33 () {56 vars in 1133 bytes} [Sun Sep 11 14:54:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:36.361373 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ce6d0-7010-494f-a7fd-333be1ab2fd7 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:36.361648 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ce6d0-7010-494f-a7fd-333be1ab2fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:36.361811 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b90a850-7e37-4420-a148-55c5457f41d4 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:36.361906 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ce6d0-7010-494f-a7fd-333be1ab2fd7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:36.372899 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9ce6d0-7010-494f-a7fd-333be1ab2fd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:36.379511 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-fc9ce6d0-7010-494f-a7fd-333be1ab2fd7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:54:36.379511 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:36.379511 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:36.379511 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:36.379511 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:36.379511 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:36.379511 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:36.379511 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:36.379511 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:36.379511 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:36.379511 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:36.379511 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:36.379511 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:36.379511 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:36.379511 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:36.379511 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:36.380504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:36.380504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:36.380504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:36.380504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:36.380504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:36.380504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:36.380504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:36.380504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:36.380504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:36.380504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:36.380504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:36.380504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:36.380504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:36.380504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:36.380504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:36.380504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:36.381355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:36.381355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:36.381355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:36.381355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:36.381355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:36.381355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:36.381355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:36.381355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:36.381355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:36.381355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:36.381355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:54:36.381355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:36.381355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 289/570] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 14:54:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:36.382036 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169c45a-7cb4-4293-a3a2-d584a7e6c7f6 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:36.382648 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169c45a-7cb4-4293-a3a2-d584a7e6c7f6 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:36.382648 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169c45a-7cb4-4293-a3a2-d584a7e6c7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:36.401571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-7169c45a-7cb4-4293-a3a2-d584a7e6c7f6 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:36.402787 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7169c45a-7cb4-4293-a3a2-d584a7e6c7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:36.466442 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b90a850-7e37-4420-a148-55c5457f41d4 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:36.470765 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 290/571] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:36.474217 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169c45a-7cb4-4293-a3a2-d584a7e6c7f6 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:36.502874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d37ca92-b8fa-40da-b701-5d8b3f082fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:36.503490 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37ca92-b8fa-40da-b701-5d8b3f082fbd None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:36.503772 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37ca92-b8fa-40da-b701-5d8b3f082fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:36.504048 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37ca92-b8fa-40da-b701-5d8b3f082fbd None admin] PATH_INFO: `/v3/projects` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:36.504739 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d37ca92-b8fa-40da-b701-5d8b3f082fbd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:36.505049 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d37ca92-b8fa-40da-b701-5d8b3f082fbd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:36.506504 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d37ca92-b8fa-40da-b701-5d8b3f082fbd None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:36.522725 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 282/572] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:54:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:36.522725 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d37ca92-b8fa-40da-b701-5d8b3f082fbd None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:36.523587 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:36.533704 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:36.588268 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169c45a-7cb4-4293-a3a2-d584a7e6c7f6 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:36.625133 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e0cee9-2e59-4e90-b02d-2d3fa87b2968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:36.625590 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d37ca92-b8fa-40da-b701-5d8b3f082fbd None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:36.626182 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0cee9-2e59-4e90-b02d-2d3fa87b2968 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:36.626613 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0cee9-2e59-4e90-b02d-2d3fa87b2968 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:36.627161 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0cee9-2e59-4e90-b02d-2d3fa87b2968 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:36.628057 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e0cee9-2e59-4e90-b02d-2d3fa87b2968 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:36.628360 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e0cee9-2e59-4e90-b02d-2d3fa87b2968 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:36.629731 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e0cee9-2e59-4e90-b02d-2d3fa87b2968 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:36.642197 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 291/573] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:36.643033 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e0cee9-2e59-4e90-b02d-2d3fa87b2968 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:36.663704 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af7d8f2-3ecd-4b7d-b27b-b2dbc6929e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:36.664334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7d8f2-3ecd-4b7d-b27b-b2dbc6929e64 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:36.664612 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7d8f2-3ecd-4b7d-b27b-b2dbc6929e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:36.664882 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7d8f2-3ecd-4b7d-b27b-b2dbc6929e64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:36.675956 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af7d8f2-3ecd-4b7d-b27b-b2dbc6929e64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:36.677407 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af7d8f2-3ecd-4b7d-b27b-b2dbc6929e64 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:36.693889 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-7af7d8f2-3ecd-4b7d-b27b-b2dbc6929e64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:36.693889 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:36.693889 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:36.693889 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:36.693889 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:36.693889 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:36.693889 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:36.693889 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:36.693889 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:36.693889 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:36.693889 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:36.693889 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:36.693889 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:36.693889 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:36.693889 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:36.693889 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:36.695055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:36.695055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:36.695055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:36.695055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:36.695055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:36.695055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:36.695055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:36.695055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:36.695055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:36.695055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:36.695055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:36.695055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:36.695055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:36.695055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:36.695055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:36.695055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:36.695055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:36.696017 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:36.696017 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:36.696017 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:36.698949 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 283/574] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:36.699760 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af7d8f2-3ecd-4b7d-b27b-b2dbc6929e64 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:36.751648 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:36.754621 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e0cee9-2e59-4e90-b02d-2d3fa87b2968 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:36.764581 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:36.794318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7106e9-bd64-4486-884c-c1e96e396171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:36.794971 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7106e9-bd64-4486-884c-c1e96e396171 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:36.795259 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7106e9-bd64-4486-884c-c1e96e396171 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:36.795534 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7106e9-bd64-4486-884c-c1e96e396171 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:36.796743 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7106e9-bd64-4486-884c-c1e96e396171 None admin] RBAC: Adding query filter params (name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:36.797127 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7106e9-bd64-4486-884c-c1e96e396171 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:36.798512 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7106e9-bd64-4486-884c-c1e96e396171 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:36.806635 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 292/575] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 14:54:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:36.807371 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af7d8f2-3ecd-4b7d-b27b-b2dbc6929e64 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:36.807480 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7106e9-bd64-4486-884c-c1e96e396171 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:36.844511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aa9ee0-7e87-40f4-818c-edfdfbb5d2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:36.845114 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa9ee0-7e87-40f4-818c-edfdfbb5d2ae None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:36.845394 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa9ee0-7e87-40f4-818c-edfdfbb5d2ae None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:36.845666 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa9ee0-7e87-40f4-818c-edfdfbb5d2ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:36.846384 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aa9ee0-7e87-40f4-818c-edfdfbb5d2ae None admin] RBAC: Adding query filter params (role.id=533fa341c4f64cd4b0a7ebb668f9106a, scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176, user.id=698f1293294d45b89e04d85649f454d0) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:36.846691 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aa9ee0-7e87-40f4-818c-edfdfbb5d2ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:36.848158 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aa9ee0-7e87-40f4-818c-edfdfbb5d2ae None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:36.853703 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 284/576] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:54:36 2022] GET /identity/v3/role_assignments?user.id=698f1293294d45b89e04d85649f454d0&scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176&role.id=533fa341c4f64cd4b0a7ebb668f9106a => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:36.854468 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aa9ee0-7e87-40f4-818c-edfdfbb5d2ae None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:36.940108 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7106e9-bd64-4486-884c-c1e96e396171 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:36.977102 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59b8c1-4517-4181-8a24-8cb5fc51fa78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:36.977773 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59b8c1-4517-4181-8a24-8cb5fc51fa78 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:36.978058 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59b8c1-4517-4181-8a24-8cb5fc51fa78 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:36.978381 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59b8c1-4517-4181-8a24-8cb5fc51fa78 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:36.978951 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aa9ee0-7e87-40f4-818c-edfdfbb5d2ae None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:36.992246 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d59b8c1-4517-4181-8a24-8cb5fc51fa78 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:36.992246 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d59b8c1-4517-4181-8a24-8cb5fc51fa78 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:37.004973 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-7d59b8c1-4517-4181-8a24-8cb5fc51fa78 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 11 14:54:37.004973 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:37.004973 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:37.004973 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:37.004973 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:37.004973 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:37.004973 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:37.004973 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:37.004973 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:37.004973 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:37.004973 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:37.004973 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:37.004973 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:54:37.004973 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:54:37.004973 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:54:37.004973 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:54:37.005959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:37.005959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:37.005959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:54:37.005959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:37.005959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:54:37.005959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:37.005959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:37.005959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:37.005959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:37.005959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:37.005959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:37.005959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:37.005959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:37.005959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:37.005959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:37.005959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:37.005959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:37.005959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:37.006935 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:37.006935 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:37.006935 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:54:37.006935 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:54:37.006935 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:54:37.006935 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:54:37.006935 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:54:37.006935 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:54:37.006935 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 11 14:54:37.006935 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:37.006935 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 293/577] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:54:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:37.006935 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59b8c1-4517-4181-8a24-8cb5fc51fa78 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:37.012139 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:37.027017 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319d9e41-c661-4597-9c6c-5a83de5812c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:37.027304 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d9e41-c661-4597-9c6c-5a83de5812c7 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:37.028311 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d9e41-c661-4597-9c6c-5a83de5812c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:37.028311 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d9e41-c661-4597-9c6c-5a83de5812c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:37.028619 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319d9e41-c661-4597-9c6c-5a83de5812c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:37.028922 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319d9e41-c661-4597-9c6c-5a83de5812c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:37.030877 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319d9e41-c661-4597-9c6c-5a83de5812c7 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:37.037105 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:37.045215 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 285/578] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:37.050080 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319d9e41-c661-4597-9c6c-5a83de5812c7 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:37.119452 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59b8c1-4517-4181-8a24-8cb5fc51fa78 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:37.155463 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144aa1d7-a1d9-442a-8110-3c4a4f029614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:37.156122 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144aa1d7-a1d9-442a-8110-3c4a4f029614 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:37.156408 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144aa1d7-a1d9-442a-8110-3c4a4f029614 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:37.156683 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144aa1d7-a1d9-442a-8110-3c4a4f029614 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:37.158182 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319d9e41-c661-4597-9c6c-5a83de5812c7 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:37.167265 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144aa1d7-a1d9-442a-8110-3c4a4f029614 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:37.173131 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-144aa1d7-a1d9-442a-8110-3c4a4f029614 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:54:37.173131 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:37.173131 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:37.173131 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:37.173131 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:37.173131 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:37.173131 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:37.173131 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:37.173131 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:37.173131 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:37.173131 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:37.173131 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:37.173131 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:37.173131 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:37.173131 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:37.173131 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:37.174244 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:37.174244 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:37.174244 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:37.174244 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:37.174244 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:37.174244 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:37.174244 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:37.174244 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:37.174244 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:37.174244 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:37.174244 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:37.174244 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:37.174244 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:37.174244 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:37.174244 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:37.174244 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:37.175307 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:37.175307 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:37.175307 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:37.175307 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:37.175307 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:37.175307 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:37.175307 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:37.175307 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:37.175307 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:37.175307 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:37.175307 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:54:37.175307 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:37.175307 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 294/579] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 14:54:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:37.176009 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144aa1d7-a1d9-442a-8110-3c4a4f029614 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:37.206512 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f096c9-160d-425b-8c05-251bdec59bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:37.207445 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f096c9-160d-425b-8c05-251bdec59bbe None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:37.207869 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f096c9-160d-425b-8c05-251bdec59bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:37.208541 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f096c9-160d-425b-8c05-251bdec59bbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:37.220164 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f096c9-160d-425b-8c05-251bdec59bbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:37.221925 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f096c9-160d-425b-8c05-251bdec59bbe None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:37.240375 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-a9f096c9-160d-425b-8c05-251bdec59bbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:37.240375 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:37.240375 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:37.240375 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:37.240375 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:37.240375 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:37.240375 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:37.240375 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:37.240375 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:37.240375 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:37.240375 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:37.240375 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:37.240375 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:37.240375 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:37.240375 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:37.240375 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:37.241498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:37.241498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:37.241498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:37.241498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:37.241498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:37.241498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:37.241498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:37.241498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:37.241498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:37.241498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:37.241498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:37.241498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:37.241498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:37.241498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:37.241498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:37.241498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:37.241498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:37.242582 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:37.242582 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:37.242582 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:37.243958 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 286/580] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:37.244875 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f096c9-160d-425b-8c05-251bdec59bbe None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:37.266091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:37.283739 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:37.290411 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144aa1d7-a1d9-442a-8110-3c4a4f029614 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:37.326455 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495edab-3e99-46b8-bb98-7e1533cda836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:37.327628 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495edab-3e99-46b8-bb98-7e1533cda836 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:37.328072 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495edab-3e99-46b8-bb98-7e1533cda836 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:37.328560 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495edab-3e99-46b8-bb98-7e1533cda836 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:37.329322 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1495edab-3e99-46b8-bb98-7e1533cda836 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:37.329627 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1495edab-3e99-46b8-bb98-7e1533cda836 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:37.331104 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1495edab-3e99-46b8-bb98-7e1533cda836 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:37.349550 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 295/581] 10.222.0.33 () {56 vars in 1137 bytes} [Sun Sep 11 14:54:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:37.350179 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495edab-3e99-46b8-bb98-7e1533cda836 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:37.361828 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f096c9-160d-425b-8c05-251bdec59bbe None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:37.398324 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a3656-2af8-427e-a81d-c5a75f083cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:37.398988 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a3656-2af8-427e-a81d-c5a75f083cda None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:37.399267 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a3656-2af8-427e-a81d-c5a75f083cda None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:37.399537 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a3656-2af8-427e-a81d-c5a75f083cda None admin] PATH_INFO: `/v3/projects/service` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:37.406824 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a3656-2af8-427e-a81d-c5a75f083cda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:37.408296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a3656-2af8-427e-a81d-c5a75f083cda None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:37.419918 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-386a3656-2af8-427e-a81d-c5a75f083cda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:37.419918 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:37.419918 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:37.419918 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:37.419918 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:37.419918 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:37.419918 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:37.419918 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:37.419918 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:37.419918 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:37.419918 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:37.419918 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:37.419918 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:54:37.419918 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:54:37.419918 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:54:37.421388 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:54:37.421388 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:37.421388 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:37.421388 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:37.421388 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:37.421388 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:37.421388 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:37.421388 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:37.421388 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:37.421388 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:37.421388 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:37.421388 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:37.421388 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:37.421388 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:37.421388 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:37.421388 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:37.421388 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:37.422393 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:54:37.422393 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:54:37.422393 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:54:37.422393 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:54:37.422393 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:54:37.422393 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:54:37.422393 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:37.422393 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:37.422393 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 287/582] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:54:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:37.422917 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a3656-2af8-427e-a81d-c5a75f083cda None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:37.463225 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495edab-3e99-46b8-bb98-7e1533cda836 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:37.500998 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec45baf-8a02-4b54-a68b-48cae46b7af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:37.501629 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec45baf-8a02-4b54-a68b-48cae46b7af8 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:37.501865 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec45baf-8a02-4b54-a68b-48cae46b7af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:37.502248 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec45baf-8a02-4b54-a68b-48cae46b7af8 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:37.502933 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec45baf-8a02-4b54-a68b-48cae46b7af8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:37.503234 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec45baf-8a02-4b54-a68b-48cae46b7af8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:37.504731 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec45baf-8a02-4b54-a68b-48cae46b7af8 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:37.518419 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 296/583] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 14:54:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:37.518419 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec45baf-8a02-4b54-a68b-48cae46b7af8 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:37.521324 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:37.527019 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:37.535563 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a3656-2af8-427e-a81d-c5a75f083cda None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:37.576028 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f88b29-0fd1-4a8a-8310-fa43005c9179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:37.576694 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f88b29-0fd1-4a8a-8310-fa43005c9179 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:37.577035 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f88b29-0fd1-4a8a-8310-fa43005c9179 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:37.577939 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f88b29-0fd1-4a8a-8310-fa43005c9179 None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:37.578574 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f88b29-0fd1-4a8a-8310-fa43005c9179 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:37.578920 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f88b29-0fd1-4a8a-8310-fa43005c9179 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:37.580351 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f88b29-0fd1-4a8a-8310-fa43005c9179 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:37.598314 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 288/584] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:37.598314 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f88b29-0fd1-4a8a-8310-fa43005c9179 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:37.633069 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec45baf-8a02-4b54-a68b-48cae46b7af8 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:37.666472 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e39973-0470-4974-bdcf-6948f65d53f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:37.667363 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e39973-0470-4974-bdcf-6948f65d53f8 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:37.667640 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e39973-0470-4974-bdcf-6948f65d53f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:37.667907 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e39973-0470-4974-bdcf-6948f65d53f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:37.678200 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e39973-0470-4974-bdcf-6948f65d53f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:37.679681 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e39973-0470-4974-bdcf-6948f65d53f8 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:37.694182 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-65e39973-0470-4974-bdcf-6948f65d53f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:37.694182 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:37.694182 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:37.694182 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:37.694182 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:37.694182 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:37.694182 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:37.694182 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:37.694182 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:37.694182 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:37.694182 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:37.694182 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:37.694182 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:37.694182 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:37.694182 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:37.694182 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:37.695224 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:37.695224 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:37.695224 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:37.695224 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:37.695224 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:37.695224 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:37.695224 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:37.695224 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:37.695224 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:37.695224 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:37.695224 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:37.695224 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:37.695224 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:37.695224 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:37.695224 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:37.695224 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:37.695224 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:37.696153 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:37.696153 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:37.696153 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:37.696153 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 297/585] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:37.696153 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e39973-0470-4974-bdcf-6948f65d53f8 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:37.713591 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f88b29-0fd1-4a8a-8310-fa43005c9179 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:37.755105 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5462dc7-d296-4abc-b10c-7d8edb5910b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:37.755824 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:37.757205 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5462dc7-d296-4abc-b10c-7d8edb5910b8 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:37.757753 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5462dc7-d296-4abc-b10c-7d8edb5910b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:37.758199 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5462dc7-d296-4abc-b10c-7d8edb5910b8 None admin] PATH_INFO: `/v3/projects` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:37.759098 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5462dc7-d296-4abc-b10c-7d8edb5910b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:37.759551 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5462dc7-d296-4abc-b10c-7d8edb5910b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:37.761133 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5462dc7-d296-4abc-b10c-7d8edb5910b8 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:37.774666 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 289/586] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:54:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:37.775711 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5462dc7-d296-4abc-b10c-7d8edb5910b8 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:37.798403 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:37.807233 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e39973-0470-4974-bdcf-6948f65d53f8 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:37.844848 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43665a0c-8612-4694-ab2e-ece616ba7070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:37.845478 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43665a0c-8612-4694-ab2e-ece616ba7070 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:37.845756 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43665a0c-8612-4694-ab2e-ece616ba7070 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:37.846030 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43665a0c-8612-4694-ab2e-ece616ba7070 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:37.856566 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43665a0c-8612-4694-ab2e-ece616ba7070 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:37.858033 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43665a0c-8612-4694-ab2e-ece616ba7070 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:37.872661 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-43665a0c-8612-4694-ab2e-ece616ba7070 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:37.872661 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:37.872661 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:37.872661 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:37.872661 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:37.872661 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:37.872661 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:37.872661 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:37.872661 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:37.872661 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:37.872661 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:37.872661 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:37.872661 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:37.872661 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:37.872661 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:37.872661 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:37.873614 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:37.873614 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:37.873614 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:37.873614 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:37.873614 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:37.873614 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:37.873614 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:37.873614 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:37.873614 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:37.873614 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:37.873614 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:37.873614 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:37.873614 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:37.873614 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:37.873614 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:37.873614 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:37.873614 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:37.875469 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:37.875469 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:37.875469 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:37.875469 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 298/587] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:37.875710 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43665a0c-8612-4694-ab2e-ece616ba7070 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:37.890733 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5462dc7-d296-4abc-b10c-7d8edb5910b8 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:37.928304 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a452e-5c6d-49f4-a246-159aea911d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:37.928933 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a452e-5c6d-49f4-a246-159aea911d13 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:37.929211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a452e-5c6d-49f4-a246-159aea911d13 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:37.929482 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a452e-5c6d-49f4-a246-159aea911d13 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:37.937720 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a452e-5c6d-49f4-a246-159aea911d13 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:37.939238 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a452e-5c6d-49f4-a246-159aea911d13 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:37.950713 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-058a452e-5c6d-49f4-a246-159aea911d13 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 14:54:37.950713 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:37.950713 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:37.950713 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:37.950713 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:37.950713 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:37.950713 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:37.950713 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:37.950713 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:37.950713 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:37.950713 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:37.950713 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:37.950713 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:54:37.950713 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:54:37.950713 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:54:37.950713 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:54:37.951719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:37.951719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:37.951719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:54:37.951719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:37.951719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:54:37.951719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:37.951719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:37.951719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:37.951719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:37.951719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:37.951719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:37.951719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:37.951719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:37.951719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:37.951719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:37.951719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:37.951719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:37.951719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:37.953239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:37.953239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:37.953239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:54:37.953239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:54:37.953239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:54:37.953239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:54:37.953239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:54:37.953239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:54:37.953239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 14:54:37.953239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:37.953239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 290/588] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:54:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:37.953857 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a452e-5c6d-49f4-a246-159aea911d13 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:37.996279 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43665a0c-8612-4694-ab2e-ece616ba7070 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:38.004146 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:38.034593 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315b1b0-76ca-4dfb-90af-5d0011e8ee0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:38.035158 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315b1b0-76ca-4dfb-90af-5d0011e8ee0a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:38.035440 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315b1b0-76ca-4dfb-90af-5d0011e8ee0a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:38.035712 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315b1b0-76ca-4dfb-90af-5d0011e8ee0a None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:38.036348 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315b1b0-76ca-4dfb-90af-5d0011e8ee0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:38.036652 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315b1b0-76ca-4dfb-90af-5d0011e8ee0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:38.038037 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315b1b0-76ca-4dfb-90af-5d0011e8ee0a None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:38.044526 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:38.051627 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 299/589] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:38.052707 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315b1b0-76ca-4dfb-90af-5d0011e8ee0a None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:38.071536 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a452e-5c6d-49f4-a246-159aea911d13 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:38.111062 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7023048-572e-43e2-8d6d-7b76988d5a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:38.111724 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7023048-572e-43e2-8d6d-7b76988d5a9f None admin] REQUEST_METHOD: `PUT` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:38.112002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7023048-572e-43e2-8d6d-7b76988d5a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:38.112287 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7023048-572e-43e2-8d6d-7b76988d5a9f None admin] PATH_INFO: `/v3/projects/9e4a3d44808e4b4092cfc4ee284c0176/users/fc0ea93084b0482fb2dd6d802987942f/roles/533fa341c4f64cd4b0a7ebb668f9106a` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:38.129773 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7023048-572e-43e2-8d6d-7b76988d5a9f None admin] RBAC: Authorizing `identity:create_grant(project_id=9e4a3d44808e4b4092cfc4ee284c0176, user_id=fc0ea93084b0482fb2dd6d802987942f, role_id=533fa341c4f64cd4b0a7ebb668f9106a)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:38.131263 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7023048-572e-43e2-8d6d-7b76988d5a9f None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:38.150886 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 291/590] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 14:54:37 2022] PUT /identity/v3/projects/9e4a3d44808e4b4092cfc4ee284c0176/users/fc0ea93084b0482fb2dd6d802987942f/roles/533fa341c4f64cd4b0a7ebb668f9106a => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:54:38.150886 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7023048-572e-43e2-8d6d-7b76988d5a9f None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:38.171538 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315b1b0-76ca-4dfb-90af-5d0011e8ee0a None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:38.213246 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba609b-5136-46dc-91d6-e8d5f99b226c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:38.214003 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba609b-5136-46dc-91d6-e8d5f99b226c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:38.214278 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba609b-5136-46dc-91d6-e8d5f99b226c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:38.214544 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba609b-5136-46dc-91d6-e8d5f99b226c None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:38.215229 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba609b-5136-46dc-91d6-e8d5f99b226c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:38.215583 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba609b-5136-46dc-91d6-e8d5f99b226c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:38.217254 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba609b-5136-46dc-91d6-e8d5f99b226c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:38.237201 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 300/591] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:38.238168 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba609b-5136-46dc-91d6-e8d5f99b226c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:38.269388 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:38.292991 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7023048-572e-43e2-8d6d-7b76988d5a9f None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:38.333957 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:38.348816 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b5115-d0f7-4098-b174-28ac2ae08bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:38.349558 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b5115-d0f7-4098-b174-28ac2ae08bb8 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:38.350112 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b5115-d0f7-4098-b174-28ac2ae08bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:38.350389 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b5115-d0f7-4098-b174-28ac2ae08bb8 None admin] PATH_INFO: `/v3/users` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:38.351161 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b5115-d0f7-4098-b174-28ac2ae08bb8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:38.351758 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b5115-d0f7-4098-b174-28ac2ae08bb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:38.353995 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b5115-d0f7-4098-b174-28ac2ae08bb8 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:38.362529 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba609b-5136-46dc-91d6-e8d5f99b226c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:38.378203 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 292/592] 10.222.0.33 () {56 vars in 1137 bytes} [Sun Sep 11 14:54:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:38.379111 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b5115-d0f7-4098-b174-28ac2ae08bb8 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:38.412554 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed993ce-0917-4690-a970-10055a2cc532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:38.413197 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed993ce-0917-4690-a970-10055a2cc532 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:38.413557 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed993ce-0917-4690-a970-10055a2cc532 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:38.414223 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed993ce-0917-4690-a970-10055a2cc532 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:38.423681 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed993ce-0917-4690-a970-10055a2cc532 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:38.426296 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed993ce-0917-4690-a970-10055a2cc532 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:38.440609 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-4ed993ce-0917-4690-a970-10055a2cc532 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:38.440609 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:38.440609 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:38.440609 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:38.440609 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:38.440609 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:38.440609 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:38.440609 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:38.440609 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:38.440609 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:38.440609 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:38.440609 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:38.440609 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:54:38.440609 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:54:38.440609 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:54:38.441528 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:54:38.441528 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:38.441528 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:38.441528 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:38.441528 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:38.441528 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:38.441528 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:38.441528 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:38.441528 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:38.441528 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:38.441528 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:38.441528 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:38.441528 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:38.441528 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:38.441528 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:38.441528 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:38.441528 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:38.442394 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:54:38.442394 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:54:38.442394 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:54:38.442394 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:54:38.442394 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:54:38.442394 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:54:38.442394 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:38.442394 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:38.444780 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 301/593] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:54:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:38.445864 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed993ce-0917-4690-a970-10055a2cc532 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:38.513783 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b5115-d0f7-4098-b174-28ac2ae08bb8 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:38.540094 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:38.558882 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386eb332-792c-4c4d-9c52-16b213d219db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:38.559898 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386eb332-792c-4c4d-9c52-16b213d219db None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:38.560380 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386eb332-792c-4c4d-9c52-16b213d219db None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:38.560855 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386eb332-792c-4c4d-9c52-16b213d219db None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:38.562195 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed993ce-0917-4690-a970-10055a2cc532 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:38.569465 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386eb332-792c-4c4d-9c52-16b213d219db None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:38.571268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386eb332-792c-4c4d-9c52-16b213d219db None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:38.584415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-386eb332-792c-4c4d-9c52-16b213d219db None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 14:54:38.584415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:38.584415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:38.584415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:38.584415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:38.584415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:38.584415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:38.584415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:38.584415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:38.584415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:38.584415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:38.584415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:38.584415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:54:38.584415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:54:38.584415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:54:38.584415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:54:38.585413 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:38.585413 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:38.585413 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:54:38.585413 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:38.585413 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:54:38.585413 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:38.585413 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:38.585413 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:38.585413 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:38.585413 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:38.585413 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:38.585413 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:38.585413 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:38.585413 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:38.585413 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:38.585413 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:38.585413 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:38.585413 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:38.586341 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:38.586341 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:38.586341 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:54:38.586341 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:54:38.586341 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:54:38.586341 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:54:38.586341 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:54:38.586341 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:54:38.586341 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 14:54:38.586341 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:38.594091 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 293/594] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 14:54:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:38.595305 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386eb332-792c-4c4d-9c52-16b213d219db None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:38.601679 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c3a72-9e6b-4c28-8abd-6523ab19c45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:38.602316 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c3a72-9e6b-4c28-8abd-6523ab19c45c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:38.602594 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c3a72-9e6b-4c28-8abd-6523ab19c45c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:38.602981 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c3a72-9e6b-4c28-8abd-6523ab19c45c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:38.617383 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4c3a72-9e6b-4c28-8abd-6523ab19c45c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:38.618889 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4c3a72-9e6b-4c28-8abd-6523ab19c45c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:38.639153 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-8e4c3a72-9e6b-4c28-8abd-6523ab19c45c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:38.639153 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:38.639153 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:38.639153 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:38.639153 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:38.639153 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:38.639153 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:38.639153 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:38.639153 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:38.639153 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:38.639153 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:38.639153 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:38.639153 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:38.639153 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:38.639153 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:38.639153 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:38.641340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:38.641340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:38.641340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:38.641340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:38.641340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:38.641340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:38.641340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:38.641340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:38.641340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:38.641340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:38.641340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:38.641340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:38.641340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:38.641340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:38.641340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:38.641340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:38.641340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:38.642420 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:38.642420 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:38.642420 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:38.642420 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 302/595] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:38.642420 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c3a72-9e6b-4c28-8abd-6523ab19c45c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:38.656346 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:38.724708 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386eb332-792c-4c4d-9c52-16b213d219db None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:38.758324 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c3a72-9e6b-4c28-8abd-6523ab19c45c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:38.769946 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fe91d4-9348-4d0d-a984-00a4b6d23637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:38.771421 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fe91d4-9348-4d0d-a984-00a4b6d23637 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:38.771863 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fe91d4-9348-4d0d-a984-00a4b6d23637 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:38.772288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fe91d4-9348-4d0d-a984-00a4b6d23637 None admin] PATH_INFO: `/v3/projects` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:38.773305 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fe91d4-9348-4d0d-a984-00a4b6d23637 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:38.774141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fe91d4-9348-4d0d-a984-00a4b6d23637 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:38.776145 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fe91d4-9348-4d0d-a984-00a4b6d23637 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:38.793758 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 294/596] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:54:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:38.795277 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fe91d4-9348-4d0d-a984-00a4b6d23637 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:38.804969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe530adb-bdc5-45ab-adcd-a5dd76a59e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:38.806405 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe530adb-bdc5-45ab-adcd-a5dd76a59e6e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:38.806927 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe530adb-bdc5-45ab-adcd-a5dd76a59e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:38.807395 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe530adb-bdc5-45ab-adcd-a5dd76a59e6e None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:38.809069 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe530adb-bdc5-45ab-adcd-a5dd76a59e6e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:38.810130 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe530adb-bdc5-45ab-adcd-a5dd76a59e6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:38.812780 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe530adb-bdc5-45ab-adcd-a5dd76a59e6e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:38.834404 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:38.851333 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 303/597] 10.222.0.33 () {56 vars in 1139 bytes} [Sun Sep 11 14:54:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:38.852059 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe530adb-bdc5-45ab-adcd-a5dd76a59e6e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:38.925250 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fe91d4-9348-4d0d-a984-00a4b6d23637 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:38.928504 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:38.975992 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb98269-2a4c-43e9-99e4-fc406191d518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:38.976567 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb98269-2a4c-43e9-99e4-fc406191d518 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:38.976842 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb98269-2a4c-43e9-99e4-fc406191d518 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:38.977113 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb98269-2a4c-43e9-99e4-fc406191d518 None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:38.977765 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb98269-2a4c-43e9-99e4-fc406191d518 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:38.978063 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb98269-2a4c-43e9-99e4-fc406191d518 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:38.979476 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb98269-2a4c-43e9-99e4-fc406191d518 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:39.005102 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 295/598] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:39.005102 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb98269-2a4c-43e9-99e4-fc406191d518 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:39.009267 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe530adb-bdc5-45ab-adcd-a5dd76a59e6e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:39.059305 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41146ae-4a5a-421e-b500-631df83800b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:39.059884 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41146ae-4a5a-421e-b500-631df83800b7 None admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:39.060157 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41146ae-4a5a-421e-b500-631df83800b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:39.060424 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41146ae-4a5a-421e-b500-631df83800b7 None admin] PATH_INFO: `/v3/projects/9e4a3d44808e4b4092cfc4ee284c0176/users/cd9c8b53871e415facea40c6694f4171/roles/533fa341c4f64cd4b0a7ebb668f9106a` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:39.077323 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41146ae-4a5a-421e-b500-631df83800b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e4a3d44808e4b4092cfc4ee284c0176, user_id=cd9c8b53871e415facea40c6694f4171, role_id=533fa341c4f64cd4b0a7ebb668f9106a)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:39.078799 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41146ae-4a5a-421e-b500-631df83800b7 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:39.094097 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 304/599] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 14:54:38 2022] PUT /identity/v3/projects/9e4a3d44808e4b4092cfc4ee284c0176/users/cd9c8b53871e415facea40c6694f4171/roles/533fa341c4f64cd4b0a7ebb668f9106a => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:54:39.096001 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41146ae-4a5a-421e-b500-631df83800b7 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:39.119638 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb98269-2a4c-43e9-99e4-fc406191d518 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:39.159359 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d630c688-1296-4b13-97e2-c8a8d1ebfff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:39.160045 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d630c688-1296-4b13-97e2-c8a8d1ebfff0 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:39.160329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d630c688-1296-4b13-97e2-c8a8d1ebfff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:39.160609 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d630c688-1296-4b13-97e2-c8a8d1ebfff0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:39.166444 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:39.174248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d630c688-1296-4b13-97e2-c8a8d1ebfff0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:39.175969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d630c688-1296-4b13-97e2-c8a8d1ebfff0 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:39.197722 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-d630c688-1296-4b13-97e2-c8a8d1ebfff0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:39.197722 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:39.197722 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:39.197722 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:39.197722 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:39.197722 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:39.197722 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:39.197722 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:39.197722 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:39.197722 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:39.197722 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:39.197722 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:39.197722 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:39.197722 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:39.197722 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:39.197722 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:39.198773 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:39.198773 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:39.198773 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:39.198773 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:39.198773 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:39.198773 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:39.198773 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:39.198773 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:39.198773 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:39.198773 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:39.198773 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:39.198773 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:39.198773 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:39.198773 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:39.198773 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:39.198773 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:39.198773 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:39.199724 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:39.199724 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:39.199724 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:39.204178 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 296/600] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:39.206324 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38a412-fbdc-4894-8690-05d60151cbb4 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:39.206615 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38a412-fbdc-4894-8690-05d60151cbb4 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:39.206929 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38a412-fbdc-4894-8690-05d60151cbb4 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:39.212923 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 297/601] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:39.212923 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b38a412-fbdc-4894-8690-05d60151cbb4 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:39.235252 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:39.251536 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41146ae-4a5a-421e-b500-631df83800b7 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:39.303016 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b6bf9-1f55-4028-b839-c19382bd9b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:39.303663 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b6bf9-1f55-4028-b839-c19382bd9b5f None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:39.303945 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b6bf9-1f55-4028-b839-c19382bd9b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:39.304219 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b6bf9-1f55-4028-b839-c19382bd9b5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:39.314235 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b6bf9-1f55-4028-b839-c19382bd9b5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:39.316225 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b6bf9-1f55-4028-b839-c19382bd9b5f None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:39.329419 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-962b6bf9-1f55-4028-b839-c19382bd9b5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:39.329419 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:39.329419 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:39.329419 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:39.329419 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:39.329419 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:39.329419 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:39.329419 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:39.329419 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:39.329419 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:39.329419 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:39.329419 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:39.329419 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:54:39.329419 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:54:39.329419 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:54:39.330308 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:54:39.330308 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:39.330308 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:39.330308 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:39.330308 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:39.330308 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:39.330308 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:39.330308 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:39.330308 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:39.330308 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:39.330308 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:39.330308 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:39.330308 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:39.330308 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:39.330308 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:39.330308 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:39.330308 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:39.331266 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:54:39.331266 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:54:39.331266 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:54:39.331266 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:54:39.331266 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:54:39.331266 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:54:39.331266 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:39.331266 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:39.331266 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 305/602] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:54:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:39.331782 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e60ce-db69-4329-8cea-8916bb1df5fa None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:39.332571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e60ce-db69-4329-8cea-8916bb1df5fa None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:39.332571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e60ce-db69-4329-8cea-8916bb1df5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:39.351493 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b38a412-fbdc-4894-8690-05d60151cbb4 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:39.360026 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-cf0e60ce-db69-4329-8cea-8916bb1df5fa None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:39.361253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-cf0e60ce-db69-4329-8cea-8916bb1df5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:39.390479 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ddf43-e35f-4723-83bc-40985d3ad3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:39.391860 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ddf43-e35f-4723-83bc-40985d3ad3ed None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:39.392503 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ddf43-e35f-4723-83bc-40985d3ad3ed None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:39.393042 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ddf43-e35f-4723-83bc-40985d3ad3ed None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:39.394234 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336ddf43-e35f-4723-83bc-40985d3ad3ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:39.394880 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336ddf43-e35f-4723-83bc-40985d3ad3ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:39.396752 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336ddf43-e35f-4723-83bc-40985d3ad3ed None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:39.411441 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 298/603] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:39.416964 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ddf43-e35f-4723-83bc-40985d3ad3ed None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:39.446483 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 306/604] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:39.447683 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0e60ce-db69-4329-8cea-8916bb1df5fa None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:39.513017 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:39.562708 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:39.604592 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ddf43-e35f-4723-83bc-40985d3ad3ed None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:39.641435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0e60ce-db69-4329-8cea-8916bb1df5fa None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:39.651547 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0d672-3e4b-4407-bee3-9c66e447a487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:39.652376 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0d672-3e4b-4407-bee3-9c66e447a487 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:39.652809 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0d672-3e4b-4407-bee3-9c66e447a487 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:39.653243 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0d672-3e4b-4407-bee3-9c66e447a487 None admin] PATH_INFO: `/v3/projects` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:39.654105 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf0d672-3e4b-4407-bee3-9c66e447a487 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:39.654560 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf0d672-3e4b-4407-bee3-9c66e447a487 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:39.656861 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf0d672-3e4b-4407-bee3-9c66e447a487 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:39.679139 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 299/605] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:54:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:39.683435 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0d672-3e4b-4407-bee3-9c66e447a487 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:39.709205 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb2586f-f036-40cb-aa94-7897eacbf7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:39.709869 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2586f-f036-40cb-aa94-7897eacbf7a1 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:39.710115 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2586f-f036-40cb-aa94-7897eacbf7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:39.710379 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2586f-f036-40cb-aa94-7897eacbf7a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:39.721925 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb2586f-f036-40cb-aa94-7897eacbf7a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:39.723432 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb2586f-f036-40cb-aa94-7897eacbf7a1 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:39.741094 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-2fb2586f-f036-40cb-aa94-7897eacbf7a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:39.741094 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:39.741094 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:39.741094 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:39.741094 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:39.741094 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:39.741094 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:39.741094 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:39.741094 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:39.741094 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:39.741094 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:39.741094 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:39.741094 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:54:39.741094 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:54:39.741094 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:54:39.741993 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:54:39.741993 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:39.741993 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:39.741993 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:39.741993 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:39.741993 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:39.741993 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:39.741993 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:39.741993 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:39.741993 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:39.741993 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:39.741993 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:39.741993 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:39.741993 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:39.741993 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:39.741993 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:39.741993 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:39.742945 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:54:39.742945 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:54:39.742945 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:54:39.742945 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:54:39.742945 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:54:39.742945 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:54:39.742945 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:39.742945 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:39.742945 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 307/606] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:54:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:39.749386 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb2586f-f036-40cb-aa94-7897eacbf7a1 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:39.831773 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0d672-3e4b-4407-bee3-9c66e447a487 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:39.856851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:39.865604 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb2586f-f036-40cb-aa94-7897eacbf7a1 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:39.872969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e6533c-0f42-4a8d-9580-f3f9adbed1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:39.874267 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6533c-0f42-4a8d-9580-f3f9adbed1c5 None admin] REQUEST_METHOD: `PUT` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:39.874696 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6533c-0f42-4a8d-9580-f3f9adbed1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:39.875062 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6533c-0f42-4a8d-9580-f3f9adbed1c5 None admin] PATH_INFO: `/v3/projects/9e4a3d44808e4b4092cfc4ee284c0176/users/c793eafcdba249a5af3ab948e9774367/roles/533fa341c4f64cd4b0a7ebb668f9106a` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:39.879595 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:39.896455 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e6533c-0f42-4a8d-9580-f3f9adbed1c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e4a3d44808e4b4092cfc4ee284c0176, user_id=c793eafcdba249a5af3ab948e9774367, role_id=533fa341c4f64cd4b0a7ebb668f9106a)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:39.898781 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e6533c-0f42-4a8d-9580-f3f9adbed1c5 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:39.909765 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c60d70-7508-41aa-ab25-fefe50f93b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:39.910550 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c60d70-7508-41aa-ab25-fefe50f93b08 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:39.910727 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c60d70-7508-41aa-ab25-fefe50f93b08 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:39.911516 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c60d70-7508-41aa-ab25-fefe50f93b08 None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:39.912530 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c60d70-7508-41aa-ab25-fefe50f93b08 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:39.912530 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c60d70-7508-41aa-ab25-fefe50f93b08 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:39.913574 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c60d70-7508-41aa-ab25-fefe50f93b08 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:39.919410 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 300/607] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 14:54:39 2022] PUT /identity/v3/projects/9e4a3d44808e4b4092cfc4ee284c0176/users/c793eafcdba249a5af3ab948e9774367/roles/533fa341c4f64cd4b0a7ebb668f9106a => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 14:54:39.920578 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e6533c-0f42-4a8d-9580-f3f9adbed1c5 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:39.943495 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 308/608] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:54:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:39.949434 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c60d70-7508-41aa-ab25-fefe50f93b08 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:40.042548 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e6533c-0f42-4a8d-9580-f3f9adbed1c5 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:40.084077 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b09275-319d-4556-8cf7-1b500a3beeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:40.084994 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b09275-319d-4556-8cf7-1b500a3beeff None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:40.085453 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b09275-319d-4556-8cf7-1b500a3beeff None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:40.085878 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b09275-319d-4556-8cf7-1b500a3beeff None admin] PATH_INFO: `/v3/roles/service` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:40.098738 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b09275-319d-4556-8cf7-1b500a3beeff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:40.104211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-c8b09275-319d-4556-8cf7-1b500a3beeff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:54:40.104211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:40.104211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:40.104211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:40.104211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:40.104211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:40.104211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:40.104211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:40.104211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:40.104211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:40.104211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:40.104211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:40.104211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:40.104211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:40.104211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:40.104211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:40.105443 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:40.105443 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:40.105443 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:40.105443 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:40.105443 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:40.105443 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:40.105443 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:40.105443 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:40.105443 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:40.105443 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:40.105443 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:40.105443 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:40.105443 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:40.105443 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:40.105443 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:40.105443 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:40.106504 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:40.106504 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:40.106504 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:40.106504 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:40.106504 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:40.106504 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:40.106504 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:40.106504 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:40.106504 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:40.106504 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:40.106504 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:54:40.106504 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:40.109081 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 301/609] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 14:54:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:40.118465 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b09275-319d-4556-8cf7-1b500a3beeff None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:40.159747 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c60d70-7508-41aa-ab25-fefe50f93b08 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:40.190565 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:40.212425 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2a46c-4670-46ca-95ee-4182b1795efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:40.213098 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2a46c-4670-46ca-95ee-4182b1795efa None admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:40.213375 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2a46c-4670-46ca-95ee-4182b1795efa None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:40.213652 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2a46c-4670-46ca-95ee-4182b1795efa None admin] PATH_INFO: `/v3/projects/9e4a3d44808e4b4092cfc4ee284c0176/users/df6eb236e80b4395a7aa77020b2f3a4f/roles/533fa341c4f64cd4b0a7ebb668f9106a` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:40.247040 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b2a46c-4670-46ca-95ee-4182b1795efa None admin] RBAC: Authorizing `identity:create_grant(project_id=9e4a3d44808e4b4092cfc4ee284c0176, user_id=df6eb236e80b4395a7aa77020b2f3a4f, role_id=533fa341c4f64cd4b0a7ebb668f9106a)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:40.248520 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b2a46c-4670-46ca-95ee-4182b1795efa None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:40.272517 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 309/610] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 14:54:39 2022] PUT /identity/v3/projects/9e4a3d44808e4b4092cfc4ee284c0176/users/df6eb236e80b4395a7aa77020b2f3a4f/roles/533fa341c4f64cd4b0a7ebb668f9106a => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 14:54:40.290217 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b09275-319d-4556-8cf7-1b500a3beeff None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:40.332235 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05758098-4a2a-43f3-9202-3b8336afa14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:40.332859 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05758098-4a2a-43f3-9202-3b8336afa14d None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:40.333201 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05758098-4a2a-43f3-9202-3b8336afa14d None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:40.333528 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05758098-4a2a-43f3-9202-3b8336afa14d None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:40.334253 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05758098-4a2a-43f3-9202-3b8336afa14d None admin] RBAC: Adding query filter params (name=service) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:40.334563 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05758098-4a2a-43f3-9202-3b8336afa14d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:40.336071 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05758098-4a2a-43f3-9202-3b8336afa14d None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:40.352844 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 302/611] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 14:54:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:54:40.359673 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2a46c-4670-46ca-95ee-4182b1795efa None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:40.369476 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:40.495308 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2a46c-4670-46ca-95ee-4182b1795efa None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:40.541743 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb70246-eca4-49e0-ae5a-d22735d62f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:40.542394 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb70246-eca4-49e0-ae5a-d22735d62f03 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:40.542673 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb70246-eca4-49e0-ae5a-d22735d62f03 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:40.542974 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb70246-eca4-49e0-ae5a-d22735d62f03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:40.554614 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb70246-eca4-49e0-ae5a-d22735d62f03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:40.556443 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb70246-eca4-49e0-ae5a-d22735d62f03 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:40.573772 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-2cb70246-eca4-49e0-ae5a-d22735d62f03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:40.573772 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:40.573772 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:40.573772 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:40.573772 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:40.573772 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:40.573772 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:40.573772 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:40.573772 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:40.573772 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:40.573772 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:40.573772 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:40.573772 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:40.573772 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:40.573772 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:40.573772 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:40.574972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:40.574972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:40.574972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:40.574972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:40.574972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:40.574972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:40.574972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:40.574972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:40.574972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:40.574972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:40.574972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:40.574972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:40.574972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:40.574972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:40.574972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:40.574972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:40.574972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:40.576060 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:40.576060 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:40.576060 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:40.576060 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 310/612] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:40.582224 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05758098-4a2a-43f3-9202-3b8336afa14d None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:40.586484 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24cba8-af7a-42f3-afc5-418973b30540 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:40.586912 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24cba8-af7a-42f3-afc5-418973b30540 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:40.587227 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24cba8-af7a-42f3-afc5-418973b30540 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:40.589009 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 311/613] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:40.598187 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffe789-0aa3-45bb-82fc-40d4a8caa1c6 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:40.598704 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffe789-0aa3-45bb-82fc-40d4a8caa1c6 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:40.599147 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffe789-0aa3-45bb-82fc-40d4a8caa1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:40.621718 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-f1ffe789-0aa3-45bb-82fc-40d4a8caa1c6 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:40.624334 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-f1ffe789-0aa3-45bb-82fc-40d4a8caa1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:40.731108 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:40.759140 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 312/614] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:54:40.764053 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05758098-4a2a-43f3-9202-3b8336afa14d None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:40.778229 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:40.818240 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca264b33-d381-4aaa-add7-0a4741615775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:40.818882 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca264b33-d381-4aaa-add7-0a4741615775 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:40.819162 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca264b33-d381-4aaa-add7-0a4741615775 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:40.819443 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca264b33-d381-4aaa-add7-0a4741615775 None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:40.820090 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca264b33-d381-4aaa-add7-0a4741615775 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:40.820386 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca264b33-d381-4aaa-add7-0a4741615775 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:40.821765 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca264b33-d381-4aaa-add7-0a4741615775 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:40.838135 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 303/615] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:40.844067 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffe789-0aa3-45bb-82fc-40d4a8caa1c6 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:40.972938 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffe789-0aa3-45bb-82fc-40d4a8caa1c6 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:41.037398 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e8a31-f8da-442f-8277-8997a8e1e401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:41.038187 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e8a31-f8da-442f-8277-8997a8e1e401 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:41.038574 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e8a31-f8da-442f-8277-8997a8e1e401 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:41.038978 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e8a31-f8da-442f-8277-8997a8e1e401 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:41.052590 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e8a31-f8da-442f-8277-8997a8e1e401 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:41.054195 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e8a31-f8da-442f-8277-8997a8e1e401 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:41.061033 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca264b33-d381-4aaa-add7-0a4741615775 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:41.071470 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-b86e8a31-f8da-442f-8277-8997a8e1e401 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 14:54:41.071470 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:41.071470 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:41.071470 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:41.071470 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:41.071470 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:41.071470 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:41.071470 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:41.071470 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:41.071470 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:41.071470 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:41.071470 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:41.071470 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:54:41.071470 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:54:41.071470 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:54:41.071470 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:54:41.072645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:41.072645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:41.072645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:54:41.072645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:41.072645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:54:41.072645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:41.072645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:41.072645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:41.072645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:41.072645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:41.072645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:41.072645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:41.072645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:41.072645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:41.072645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:41.072645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:41.072645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:41.072645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:41.073784 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:41.073784 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:41.073784 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:54:41.073784 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:54:41.073784 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:54:41.073784 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:54:41.073784 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:54:41.073784 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:54:41.073784 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 14:54:41.073784 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:41.077964 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 313/616] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:54:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:41.081580 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e8a31-f8da-442f-8277-8997a8e1e401 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:41.159229 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:41.255897 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca264b33-d381-4aaa-add7-0a4741615775 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:41.270969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e8a31-f8da-442f-8277-8997a8e1e401 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:41.326742 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cec2b6-fcba-40f9-92e1-cfb1c6b82763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:41.327555 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cec2b6-fcba-40f9-92e1-cfb1c6b82763 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:41.327838 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cec2b6-fcba-40f9-92e1-cfb1c6b82763 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:41.328113 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cec2b6-fcba-40f9-92e1-cfb1c6b82763 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:41.328856 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cec2b6-fcba-40f9-92e1-cfb1c6b82763 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:41.329159 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cec2b6-fcba-40f9-92e1-cfb1c6b82763 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:41.330602 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cec2b6-fcba-40f9-92e1-cfb1c6b82763 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:41.336350 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce88a49-6fa3-450a-8c6f-5255175834ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:41.336993 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce88a49-6fa3-450a-8c6f-5255175834ab None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:41.337271 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce88a49-6fa3-450a-8c6f-5255175834ab None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:41.337542 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce88a49-6fa3-450a-8c6f-5255175834ab None admin] PATH_INFO: `/v3/roles/service` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:41.359405 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce88a49-6fa3-450a-8c6f-5255175834ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:41.364273 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-5ce88a49-6fa3-450a-8c6f-5255175834ab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:54:41.364273 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:41.364273 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:41.364273 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:41.364273 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:41.364273 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:41.364273 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:41.364273 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:41.364273 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:41.364273 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:41.364273 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:41.364273 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:41.364273 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:41.364273 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:41.364273 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:41.364273 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:41.365191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:41.365191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:41.365191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:41.365191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:41.365191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:41.365191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:41.365191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:41.365191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:41.365191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:41.365191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:41.365191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:41.365191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:41.365191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:41.365191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:41.365191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:41.365191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:41.366049 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:41.366049 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:41.366049 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:41.366049 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:41.366049 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:41.366049 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:41.366049 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:41.366049 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:41.366049 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:41.366049 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:41.366049 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:54:41.366049 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:41.369039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 304/617] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 14:54:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:54:41.371160 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 314/618] 10.222.0.33 () {56 vars in 1135 bytes} [Sun Sep 11 14:54:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:41.373028 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce88a49-6fa3-450a-8c6f-5255175834ab None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:41.376848 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cec2b6-fcba-40f9-92e1-cfb1c6b82763 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:41.457348 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:41.505558 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cec2b6-fcba-40f9-92e1-cfb1c6b82763 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:41.531276 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:41.544666 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75c3b3-0901-4af0-b3a8-11fea8bd7530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:41.545543 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75c3b3-0901-4af0-b3a8-11fea8bd7530 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:41.546544 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75c3b3-0901-4af0-b3a8-11fea8bd7530 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:41.554830 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75c3b3-0901-4af0-b3a8-11fea8bd7530 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:41.571086 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c75c3b3-0901-4af0-b3a8-11fea8bd7530 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:41.572566 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c75c3b3-0901-4af0-b3a8-11fea8bd7530 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:41.605459 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce88a49-6fa3-450a-8c6f-5255175834ab None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:41.613231 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-2c75c3b3-0901-4af0-b3a8-11fea8bd7530 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:41.613231 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:41.613231 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:41.613231 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:41.613231 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:41.613231 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:41.613231 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:41.613231 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:41.613231 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:41.613231 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:41.613231 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:41.613231 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:41.613231 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:41.613231 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:41.613231 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:41.613231 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:41.614619 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:41.614619 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:41.614619 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:41.614619 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:41.614619 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:41.614619 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:41.614619 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:41.614619 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:41.614619 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:41.614619 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:41.614619 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:41.614619 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:41.614619 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:41.614619 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:41.614619 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:41.614619 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:41.614619 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:41.615788 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:41.615788 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:41.615788 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:41.615788 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 315/619] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:54:41.623962 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75c3b3-0901-4af0-b3a8-11fea8bd7530 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:41.654836 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e86cd9d-029c-4c5b-baa6-e4998f0fda6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:41.656254 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86cd9d-029c-4c5b-baa6-e4998f0fda6d None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:41.656935 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86cd9d-029c-4c5b-baa6-e4998f0fda6d None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:41.657214 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86cd9d-029c-4c5b-baa6-e4998f0fda6d None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:41.659437 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e86cd9d-029c-4c5b-baa6-e4998f0fda6d None admin] RBAC: Adding query filter params (name=service) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:41.659744 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e86cd9d-029c-4c5b-baa6-e4998f0fda6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:41.661133 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e86cd9d-029c-4c5b-baa6-e4998f0fda6d None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:41.677284 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 305/620] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 14:54:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:54:41.683655 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e86cd9d-029c-4c5b-baa6-e4998f0fda6d None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:41.799143 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75c3b3-0901-4af0-b3a8-11fea8bd7530 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:41.810418 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:41.821840 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e86cd9d-029c-4c5b-baa6-e4998f0fda6d None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:41.839209 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e008bc4-a4b0-4f3b-abdb-ed217f80dad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:41.840159 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e008bc4-a4b0-4f3b-abdb-ed217f80dad0 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:41.840689 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e008bc4-a4b0-4f3b-abdb-ed217f80dad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:41.841110 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e008bc4-a4b0-4f3b-abdb-ed217f80dad0 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:41.841931 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e008bc4-a4b0-4f3b-abdb-ed217f80dad0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:41.842385 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e008bc4-a4b0-4f3b-abdb-ed217f80dad0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:41.844143 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e008bc4-a4b0-4f3b-abdb-ed217f80dad0 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:41.857855 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 316/621] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:41.859093 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5491a-c07a-4b75-9ef7-335292103e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:41.859824 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5491a-c07a-4b75-9ef7-335292103e3e None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:41.860151 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5491a-c07a-4b75-9ef7-335292103e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:41.860430 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5491a-c07a-4b75-9ef7-335292103e3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:41.860637 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25206089-4605-44e5-89ab-601bb33b5041 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:41.861236 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25206089-4605-44e5-89ab-601bb33b5041 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:41.861674 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25206089-4605-44e5-89ab-601bb33b5041 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:41.866968 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 317/622] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:41.869185 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23780b48-f459-4573-9118-74d9b1dc92a0 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:41.869640 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23780b48-f459-4573-9118-74d9b1dc92a0 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:41.870165 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23780b48-f459-4573-9118-74d9b1dc92a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:41.872273 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c5491a-c07a-4b75-9ef7-335292103e3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:41.874441 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c5491a-c07a-4b75-9ef7-335292103e3e None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:41.890169 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-23780b48-f459-4573-9118-74d9b1dc92a0 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:41.890750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-63c5491a-c07a-4b75-9ef7-335292103e3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:41.890750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:41.890750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:41.890750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:41.890750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:41.890750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:41.890750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:41.890750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:41.890750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:41.890750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:41.890750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:41.890750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:41.890750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:41.890750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:41.890750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:41.890750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:41.891852 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:41.891852 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:41.891852 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:41.891852 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:41.891852 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:41.891852 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:41.891852 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:41.891852 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:41.891852 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:41.891852 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:41.891852 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:41.891852 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:41.891852 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:41.891852 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:41.891852 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:41.891852 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:41.891852 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:41.892755 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:41.892755 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:41.892755 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:41.892755 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 306/623] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:41.893145 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5491a-c07a-4b75-9ef7-335292103e3e None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:41.894109 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-23780b48-f459-4573-9118-74d9b1dc92a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:41.909117 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:41.978920 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 318/624] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:54:41.980160 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23780b48-f459-4573-9118-74d9b1dc92a0 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:42.009955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5491a-c07a-4b75-9ef7-335292103e3e None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:42.048157 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b3607-2d43-4f54-bedd-a60e545b09b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:42.048848 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b3607-2d43-4f54-bedd-a60e545b09b4 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:42.049132 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b3607-2d43-4f54-bedd-a60e545b09b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:42.049506 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b3607-2d43-4f54-bedd-a60e545b09b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:42.057138 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b3607-2d43-4f54-bedd-a60e545b09b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:42.058945 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b3607-2d43-4f54-bedd-a60e545b09b4 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:42.071644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-651b3607-2d43-4f54-bedd-a60e545b09b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:42.071644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:42.071644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:42.071644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:42.071644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:42.071644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:42.071644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:42.071644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:42.071644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:42.071644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:42.071644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:42.071644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:42.071644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:54:42.071644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:54:42.071644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:54:42.072753 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:54:42.072753 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:42.072753 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:42.072753 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:42.072753 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:42.072753 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:42.072753 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:42.072753 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:42.072753 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:42.072753 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:42.072753 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:42.072753 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:42.072753 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:42.072753 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:42.072753 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:42.072753 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:42.072753 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:42.073787 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:54:42.073787 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:54:42.073787 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:54:42.073787 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:54:42.073787 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:54:42.073787 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:54:42.073787 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:42.073787 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:42.073787 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 307/625] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:54:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:42.078351 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b3607-2d43-4f54-bedd-a60e545b09b4 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:42.091142 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:42.189762 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b3607-2d43-4f54-bedd-a60e545b09b4 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:42.227086 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9de7f4-bff8-4550-aaaa-fae616b9e343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:42.227700 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9de7f4-bff8-4550-aaaa-fae616b9e343 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:42.227983 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9de7f4-bff8-4550-aaaa-fae616b9e343 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:42.228255 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9de7f4-bff8-4550-aaaa-fae616b9e343 None admin] PATH_INFO: `/v3/projects` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:42.228936 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9de7f4-bff8-4550-aaaa-fae616b9e343 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:42.229240 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9de7f4-bff8-4550-aaaa-fae616b9e343 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:42.230679 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9de7f4-bff8-4550-aaaa-fae616b9e343 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:42.236715 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23780b48-f459-4573-9118-74d9b1dc92a0 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:42.244644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 308/626] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:54:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:42.252433 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9de7f4-bff8-4550-aaaa-fae616b9e343 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:42.308565 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38351dd1-5e06-4a19-9952-1e1b80f3a402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:42.309301 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38351dd1-5e06-4a19-9952-1e1b80f3a402 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:42.309580 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38351dd1-5e06-4a19-9952-1e1b80f3a402 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:42.309852 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38351dd1-5e06-4a19-9952-1e1b80f3a402 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:42.310498 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38351dd1-5e06-4a19-9952-1e1b80f3a402 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:42.310796 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38351dd1-5e06-4a19-9952-1e1b80f3a402 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:42.312993 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38351dd1-5e06-4a19-9952-1e1b80f3a402 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:42.329848 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:42.331285 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 319/627] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:42.332166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38351dd1-5e06-4a19-9952-1e1b80f3a402 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:42.348308 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:42.383133 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9de7f4-bff8-4550-aaaa-fae616b9e343 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:42.431330 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c9397-5d60-48a5-8ced-db3958b4a667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:42.432317 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c9397-5d60-48a5-8ced-db3958b4a667 None admin] REQUEST_METHOD: `PUT` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:42.432602 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c9397-5d60-48a5-8ced-db3958b4a667 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:42.432882 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c9397-5d60-48a5-8ced-db3958b4a667 None admin] PATH_INFO: `/v3/projects/9e4a3d44808e4b4092cfc4ee284c0176/users/698f1293294d45b89e04d85649f454d0/roles/533fa341c4f64cd4b0a7ebb668f9106a` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:42.451115 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38351dd1-5e06-4a19-9952-1e1b80f3a402 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:42.455027 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c9397-5d60-48a5-8ced-db3958b4a667 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e4a3d44808e4b4092cfc4ee284c0176, user_id=698f1293294d45b89e04d85649f454d0, role_id=533fa341c4f64cd4b0a7ebb668f9106a)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:42.456502 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c9397-5d60-48a5-8ced-db3958b4a667 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:42.474734 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 309/628] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 14:54:42 2022] PUT /identity/v3/projects/9e4a3d44808e4b4092cfc4ee284c0176/users/698f1293294d45b89e04d85649f454d0/roles/533fa341c4f64cd4b0a7ebb668f9106a => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:54:42.475969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c9397-5d60-48a5-8ced-db3958b4a667 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:42.491025 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6568d524-9b62-4ce3-9467-4975975f7330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:42.491666 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568d524-9b62-4ce3-9467-4975975f7330 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:42.491944 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568d524-9b62-4ce3-9467-4975975f7330 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:42.492274 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568d524-9b62-4ce3-9467-4975975f7330 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:42.505293 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6568d524-9b62-4ce3-9467-4975975f7330 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:42.511415 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-6568d524-9b62-4ce3-9467-4975975f7330 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:54:42.511415 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:42.511415 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:42.511415 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:42.511415 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:42.511415 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:42.511415 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:42.511415 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:42.511415 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:42.511415 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:42.511415 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:42.511415 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:42.511415 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:42.511415 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:42.511415 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:42.511415 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:42.512504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:42.512504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:42.512504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:42.512504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:42.512504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:42.512504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:42.512504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:42.512504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:42.512504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:42.512504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:42.512504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:42.512504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:42.512504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:42.512504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:42.512504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:42.512504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:42.515754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:42.515754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:42.515754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:42.515754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:42.515754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:42.515754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:42.515754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:42.515754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:42.515754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:42.515754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:42.515754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:54:42.515754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:42.515754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 320/629] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 14:54:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:42.527907 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6568d524-9b62-4ce3-9467-4975975f7330 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:42.633869 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:42.636781 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c9397-5d60-48a5-8ced-db3958b4a667 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:42.657633 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:42.676417 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6568d524-9b62-4ce3-9467-4975975f7330 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:42.686571 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45945d1b-39a6-4380-bb50-6bc1426fa710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:42.687273 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45945d1b-39a6-4380-bb50-6bc1426fa710 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:42.687557 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45945d1b-39a6-4380-bb50-6bc1426fa710 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:42.687830 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45945d1b-39a6-4380-bb50-6bc1426fa710 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:42.696349 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45945d1b-39a6-4380-bb50-6bc1426fa710 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:42.697853 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45945d1b-39a6-4380-bb50-6bc1426fa710 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:42.711587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-45945d1b-39a6-4380-bb50-6bc1426fa710 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 14:54:42.711587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:42.711587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:42.711587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:42.711587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:42.711587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:42.711587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:42.711587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:42.711587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:42.711587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:42.711587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:42.711587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:42.711587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:54:42.711587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:54:42.711587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:54:42.711587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:54:42.712572 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:42.712572 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:42.712572 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:54:42.712572 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:42.712572 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:54:42.712572 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:42.712572 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:42.712572 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:42.712572 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:42.712572 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:42.712572 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:42.712572 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:42.712572 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:42.712572 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:42.712572 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:42.712572 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:42.712572 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:42.712572 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:42.713550 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:42.713550 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:42.713550 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:54:42.713550 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:54:42.713550 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:54:42.713550 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:54:42.713550 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:54:42.713550 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:54:42.713550 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 14:54:42.713550 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:42.713550 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 310/630] 10.222.0.33 () {56 vars in 1093 bytes} [Sun Sep 11 14:54:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:42.714715 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45530b2f-32b9-4943-9fde-d3bc67307244 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:42.714715 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45530b2f-32b9-4943-9fde-d3bc67307244 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:42.714896 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45530b2f-32b9-4943-9fde-d3bc67307244 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:42.716279 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 311/631] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:42.719792 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45530b2f-32b9-4943-9fde-d3bc67307244 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:42.720849 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cbd291-9123-43a9-b602-7d1425a01e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:42.721070 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbd291-9123-43a9-b602-7d1425a01e9f None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:42.721292 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbd291-9123-43a9-b602-7d1425a01e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:42.721563 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbd291-9123-43a9-b602-7d1425a01e9f None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:42.722364 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cbd291-9123-43a9-b602-7d1425a01e9f None admin] RBAC: Adding query filter params (name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:42.722612 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cbd291-9123-43a9-b602-7d1425a01e9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:42.724049 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cbd291-9123-43a9-b602-7d1425a01e9f None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:42.732939 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 321/632] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 14:54:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:42.734982 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842bdf0-88a5-4a4e-bb0e-f7d169537640 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:42.735270 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842bdf0-88a5-4a4e-bb0e-f7d169537640 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:42.735547 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842bdf0-88a5-4a4e-bb0e-f7d169537640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:42.754213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-2842bdf0-88a5-4a4e-bb0e-f7d169537640 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:42.755447 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-2842bdf0-88a5-4a4e-bb0e-f7d169537640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:42.852920 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 322/633] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:42.854305 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842bdf0-88a5-4a4e-bb0e-f7d169537640 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:42.866655 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45530b2f-32b9-4943-9fde-d3bc67307244 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:42.926241 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f8cbf-fd7a-4874-a5d3-74138789be52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:42.926769 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f8cbf-fd7a-4874-a5d3-74138789be52 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:42.927079 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f8cbf-fd7a-4874-a5d3-74138789be52 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:42.927350 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f8cbf-fd7a-4874-a5d3-74138789be52 None admin] PATH_INFO: `/v3/users` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:42.928329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6f8cbf-fd7a-4874-a5d3-74138789be52 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:42.928638 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6f8cbf-fd7a-4874-a5d3-74138789be52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:42.930142 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6f8cbf-fd7a-4874-a5d3-74138789be52 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:42.953018 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 312/634] 10.222.0.33 () {56 vars in 1133 bytes} [Sun Sep 11 14:54:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:42.964121 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f8cbf-fd7a-4874-a5d3-74138789be52 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:42.986311 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:43.009037 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:43.032523 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842bdf0-88a5-4a4e-bb0e-f7d169537640 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:43.069443 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3967767-3907-4895-8e42-1aecb81d0aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:43.070082 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3967767-3907-4895-8e42-1aecb81d0aa1 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:43.070361 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3967767-3907-4895-8e42-1aecb81d0aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:43.070634 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3967767-3907-4895-8e42-1aecb81d0aa1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:43.081391 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3967767-3907-4895-8e42-1aecb81d0aa1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:43.082928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3967767-3907-4895-8e42-1aecb81d0aa1 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:43.088848 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f8cbf-fd7a-4874-a5d3-74138789be52 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:43.099834 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-f3967767-3907-4895-8e42-1aecb81d0aa1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:43.099834 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:43.099834 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:43.099834 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:43.099834 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:43.099834 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:43.099834 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:43.099834 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:43.099834 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:43.099834 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:43.099834 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:43.099834 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:43.099834 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:43.099834 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:43.099834 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:43.099834 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:43.101018 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:43.101018 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:43.101018 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:43.101018 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:43.101018 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:43.101018 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:43.101018 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:43.101018 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:43.101018 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:43.101018 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:43.101018 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:43.101018 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:43.101018 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:43.101018 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:43.101018 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:43.101018 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:43.101018 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:43.102073 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:43.102073 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:43.102073 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:43.102073 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 323/635] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:43.106631 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeead031-5633-4426-9bc1-71492d379fd5 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:43.106631 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeead031-5633-4426-9bc1-71492d379fd5 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:43.106631 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeead031-5633-4426-9bc1-71492d379fd5 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:43.109146 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 324/636] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:43.109967 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeead031-5633-4426-9bc1-71492d379fd5 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:43.130271 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7abbbcd-34db-4987-8cd7-93f6a02fb7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:43.131000 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7abbbcd-34db-4987-8cd7-93f6a02fb7df None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:43.131349 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7abbbcd-34db-4987-8cd7-93f6a02fb7df None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:43.131689 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7abbbcd-34db-4987-8cd7-93f6a02fb7df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:43.202460 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7abbbcd-34db-4987-8cd7-93f6a02fb7df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:43.204300 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7abbbcd-34db-4987-8cd7-93f6a02fb7df None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:43.235224 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:43.235916 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeead031-5633-4426-9bc1-71492d379fd5 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:43.250760 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:43.274721 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff9f59-049c-45cd-9048-2fda57e762bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:43.275493 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff9f59-049c-45cd-9048-2fda57e762bc None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:43.275883 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff9f59-049c-45cd-9048-2fda57e762bc None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:43.276220 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff9f59-049c-45cd-9048-2fda57e762bc None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:43.276875 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eff9f59-049c-45cd-9048-2fda57e762bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:43.277267 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eff9f59-049c-45cd-9048-2fda57e762bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:43.278682 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eff9f59-049c-45cd-9048-2fda57e762bc None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:43.296164 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 325/637] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:43.297761 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca9d57-3988-4dc2-83e4-827e492f5673 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:43.298049 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca9d57-3988-4dc2-83e4-827e492f5673 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:43.298327 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca9d57-3988-4dc2-83e4-827e492f5673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:43.317438 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 313/638] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:54:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 14:54:43.317438 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7abbbcd-34db-4987-8cd7-93f6a02fb7df None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:43.318738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-fdca9d57-3988-4dc2-83e4-827e492f5673 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:43.320088 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-fdca9d57-3988-4dc2-83e4-827e492f5673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:43.402831 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 326/639] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:43.402831 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca9d57-3988-4dc2-83e4-827e492f5673 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:43.424338 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7abbbcd-34db-4987-8cd7-93f6a02fb7df None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:43.461073 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:43.471456 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691db82-7a2c-41c4-b677-d68eec03f4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:43.472662 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691db82-7a2c-41c4-b677-d68eec03f4b9 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:43.472952 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691db82-7a2c-41c4-b677-d68eec03f4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:43.473507 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691db82-7a2c-41c4-b677-d68eec03f4b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:43.488409 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a691db82-7a2c-41c4-b677-d68eec03f4b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:43.495066 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-a691db82-7a2c-41c4-b677-d68eec03f4b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:54:43.495066 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:43.495066 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:43.495066 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:43.495066 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:43.495066 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:43.495066 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:43.495066 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:43.495066 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:43.495066 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:43.495066 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:43.495066 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:43.495066 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:43.495066 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:43.495066 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:43.495066 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:43.496007 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:43.496007 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:43.496007 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:43.496007 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:43.496007 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:43.496007 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:43.496007 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:43.496007 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:43.496007 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:43.496007 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:43.496007 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:43.496007 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:43.496007 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:43.496007 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:43.496007 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:43.496007 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:43.496924 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:43.496924 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:43.496924 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:43.496924 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:43.496924 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:43.496924 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:43.496924 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:43.496924 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:43.496924 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:43.496924 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:43.496924 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:54:43.496924 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:43.499467 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 314/640] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 14:54:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:43.500425 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691db82-7a2c-41c4-b677-d68eec03f4b9 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:43.521900 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca9d57-3988-4dc2-83e4-827e492f5673 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:43.523821 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:43.563512 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269ef55-0b19-4129-9f8e-3381cf7b3a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:43.564317 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269ef55-0b19-4129-9f8e-3381cf7b3a1e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:43.564687 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269ef55-0b19-4129-9f8e-3381cf7b3a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:43.565055 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269ef55-0b19-4129-9f8e-3381cf7b3a1e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:43.573394 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8269ef55-0b19-4129-9f8e-3381cf7b3a1e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:43.575129 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8269ef55-0b19-4129-9f8e-3381cf7b3a1e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:43.588006 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-8269ef55-0b19-4129-9f8e-3381cf7b3a1e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 11 14:54:43.588006 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:43.588006 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:43.588006 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:43.588006 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:43.588006 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:43.588006 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:43.588006 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:43.588006 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:43.588006 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:43.588006 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:43.588006 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:43.588006 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:54:43.588006 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:54:43.588006 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:54:43.588006 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:54:43.589169 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:43.589169 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:43.589169 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:54:43.589169 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:43.589169 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:54:43.589169 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:43.589169 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:43.589169 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:43.589169 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:43.589169 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:43.589169 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:43.589169 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:43.589169 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:43.589169 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:43.589169 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:43.589169 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:43.589169 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:43.589169 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:43.590369 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:43.590369 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:43.590369 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:54:43.590369 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:54:43.590369 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:54:43.590369 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:54:43.590369 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:54:43.590369 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:54:43.590369 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 11 14:54:43.590369 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:43.596930 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 327/641] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:54:43 2022] GET /identity/v3/users/glance => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:43.597968 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269ef55-0b19-4129-9f8e-3381cf7b3a1e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:43.643783 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691db82-7a2c-41c4-b677-d68eec03f4b9 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:43.691007 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb03aa6-9e18-4ba6-a70e-6fd1bd9572cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:43.691637 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb03aa6-9e18-4ba6-a70e-6fd1bd9572cd None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:43.691913 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb03aa6-9e18-4ba6-a70e-6fd1bd9572cd None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:43.692185 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb03aa6-9e18-4ba6-a70e-6fd1bd9572cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:43.703035 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb03aa6-9e18-4ba6-a70e-6fd1bd9572cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:43.704502 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb03aa6-9e18-4ba6-a70e-6fd1bd9572cd None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:43.707399 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269ef55-0b19-4129-9f8e-3381cf7b3a1e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:43.719484 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-edb03aa6-9e18-4ba6-a70e-6fd1bd9572cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:43.719484 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:43.719484 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:43.719484 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:43.719484 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:43.719484 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:43.719484 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:43.719484 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:43.719484 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:43.719484 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:43.719484 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:43.719484 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:43.719484 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:43.719484 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:43.719484 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:43.719484 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:43.720664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:43.720664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:43.720664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:43.720664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:43.720664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:43.720664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:43.720664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:43.720664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:43.720664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:43.720664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:43.720664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:43.720664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:43.720664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:43.720664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:43.720664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:43.720664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:43.720664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:43.721580 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:43.721580 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:43.721580 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:43.721580 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 315/642] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:43.721580 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb03aa6-9e18-4ba6-a70e-6fd1bd9572cd None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:43.747302 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ddeef9-b51f-441a-ac20-8173620608a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:43.747995 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ddeef9-b51f-441a-ac20-8173620608a7 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:43.748272 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ddeef9-b51f-441a-ac20-8173620608a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:43.748566 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ddeef9-b51f-441a-ac20-8173620608a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:43.749475 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ddeef9-b51f-441a-ac20-8173620608a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:43.749775 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ddeef9-b51f-441a-ac20-8173620608a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:43.751375 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ddeef9-b51f-441a-ac20-8173620608a7 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:43.759590 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 328/643] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 14:54:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:43.760202 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ddeef9-b51f-441a-ac20-8173620608a7 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:43.775035 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:43.788206 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:43.837466 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb03aa6-9e18-4ba6-a70e-6fd1bd9572cd None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:43.869867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ddeef9-b51f-441a-ac20-8173620608a7 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:43.873503 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0187483a-f24b-4c73-b657-341ffddedd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:43.875019 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187483a-f24b-4c73-b657-341ffddedd2c None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:43.875505 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187483a-f24b-4c73-b657-341ffddedd2c None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:43.875999 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187483a-f24b-4c73-b657-341ffddedd2c None admin] PATH_INFO: `/v3/users` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:43.876932 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0187483a-f24b-4c73-b657-341ffddedd2c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:43.877437 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0187483a-f24b-4c73-b657-341ffddedd2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:43.879319 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0187483a-f24b-4c73-b657-341ffddedd2c None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:43.899555 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 316/644] 10.222.0.33 () {56 vars in 1137 bytes} [Sun Sep 11 14:54:43 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:43.900679 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0187483a-f24b-4c73-b657-341ffddedd2c None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:43.909858 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e5794b-908a-49c5-a1d7-452ff7fc6ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:43.910488 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5794b-908a-49c5-a1d7-452ff7fc6ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:43.910773 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5794b-908a-49c5-a1d7-452ff7fc6ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:43.911090 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5794b-908a-49c5-a1d7-452ff7fc6ba5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:43.921667 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e5794b-908a-49c5-a1d7-452ff7fc6ba5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:43.926999 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-b0e5794b-908a-49c5-a1d7-452ff7fc6ba5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:54:43.926999 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:43.926999 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:43.926999 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:43.926999 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:43.926999 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:43.926999 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:43.926999 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:43.926999 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:43.926999 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:43.926999 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:43.926999 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:43.926999 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:43.926999 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:43.926999 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:43.926999 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:43.927928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:43.927928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:43.927928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:43.927928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:43.927928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:43.927928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:43.927928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:43.927928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:43.927928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:43.927928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:43.927928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:43.927928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:43.927928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:43.927928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:43.927928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:43.927928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:43.928950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:43.928950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:43.928950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:43.928950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:43.928950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:43.928950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:43.928950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:43.928950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:43.928950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:43.928950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:43.928950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:54:43.928950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:43.928950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 329/645] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 14:54:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:43.928950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e5794b-908a-49c5-a1d7-452ff7fc6ba5 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:44.017010 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:44.019872 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0187483a-f24b-4c73-b657-341ffddedd2c None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:44.022349 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:44.039383 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e5794b-908a-49c5-a1d7-452ff7fc6ba5 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:44.057667 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a8f71-4af0-4fa3-8e3a-d0a3c083616e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:44.058528 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a8f71-4af0-4fa3-8e3a-d0a3c083616e None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:44.058825 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a8f71-4af0-4fa3-8e3a-d0a3c083616e None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:44.059114 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a8f71-4af0-4fa3-8e3a-d0a3c083616e None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:44.059780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a8f71-4af0-4fa3-8e3a-d0a3c083616e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:44.060085 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a8f71-4af0-4fa3-8e3a-d0a3c083616e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:44.061633 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a8f71-4af0-4fa3-8e3a-d0a3c083616e None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:44.072802 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 317/646] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:44.073588 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a8f71-4af0-4fa3-8e3a-d0a3c083616e None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:44.081164 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0564c6eb-3bf9-4a9b-9b2c-11d81d7e3f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:44.081928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564c6eb-3bf9-4a9b-9b2c-11d81d7e3f25 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:44.082303 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564c6eb-3bf9-4a9b-9b2c-11d81d7e3f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:44.082675 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564c6eb-3bf9-4a9b-9b2c-11d81d7e3f25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:44.094751 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0564c6eb-3bf9-4a9b-9b2c-11d81d7e3f25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:44.096350 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0564c6eb-3bf9-4a9b-9b2c-11d81d7e3f25 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:44.112064 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-0564c6eb-3bf9-4a9b-9b2c-11d81d7e3f25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:44.112064 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:44.112064 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:44.112064 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:44.112064 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:44.112064 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:44.112064 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:44.112064 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:44.112064 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:44.112064 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:44.112064 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:44.112064 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:44.112064 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:44.112064 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:44.112064 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:44.112064 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:44.113198 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:44.113198 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:44.113198 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:44.113198 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:44.113198 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:44.113198 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:44.113198 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:44.113198 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:44.113198 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:44.113198 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:44.113198 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:44.113198 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:44.113198 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:44.113198 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:44.113198 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:44.113198 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:44.113198 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:44.114275 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:44.114275 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:44.114275 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:44.119863 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 330/647] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:44.119863 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0564c6eb-3bf9-4a9b-9b2c-11d81d7e3f25 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:44.179845 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a8f71-4af0-4fa3-8e3a-d0a3c083616e None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:44.215055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf2b25-8f40-4eb0-8b94-d4730c233812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:44.215769 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf2b25-8f40-4eb0-8b94-d4730c233812 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:44.216057 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf2b25-8f40-4eb0-8b94-d4730c233812 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:44.216334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf2b25-8f40-4eb0-8b94-d4730c233812 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:44.225866 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0564c6eb-3bf9-4a9b-9b2c-11d81d7e3f25 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:44.248152 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:44.262910 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2097929-cc70-4994-be51-e8f0ed56ee68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:44.263606 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2097929-cc70-4994-be51-e8f0ed56ee68 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:44.263955 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2097929-cc70-4994-be51-e8f0ed56ee68 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:44.264822 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2097929-cc70-4994-be51-e8f0ed56ee68 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:44.265591 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2097929-cc70-4994-be51-e8f0ed56ee68 None admin] RBAC: Adding query filter params (name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:44.266051 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2097929-cc70-4994-be51-e8f0ed56ee68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:44.267465 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2097929-cc70-4994-be51-e8f0ed56ee68 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:44.269782 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:44.278641 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 331/648] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 14:54:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:44.279560 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2097929-cc70-4994-be51-e8f0ed56ee68 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:44.298709 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf2b25-8f40-4eb0-8b94-d4730c233812 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:44.300235 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf2b25-8f40-4eb0-8b94-d4730c233812 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:44.386487 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2097929-cc70-4994-be51-e8f0ed56ee68 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:44.410566 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 318/649] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:54:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:54:44.411455 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf2b25-8f40-4eb0-8b94-d4730c233812 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:44.430949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f87385-d2c4-4de2-89a3-690a6afe1726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:44.431582 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f87385-d2c4-4de2-89a3-690a6afe1726 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:44.431865 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f87385-d2c4-4de2-89a3-690a6afe1726 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:44.432139 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f87385-d2c4-4de2-89a3-690a6afe1726 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:44.439809 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f87385-d2c4-4de2-89a3-690a6afe1726 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:44.441257 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f87385-d2c4-4de2-89a3-690a6afe1726 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:44.454017 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-12f87385-d2c4-4de2-89a3-690a6afe1726 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:44.454017 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:44.454017 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:44.454017 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:44.454017 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:44.454017 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:44.454017 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:44.454017 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:44.454017 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:44.454017 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:44.454017 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:44.454017 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:44.454017 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:54:44.454017 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:54:44.454017 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:54:44.454959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:54:44.454959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:44.454959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:44.454959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:44.454959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:44.454959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:44.454959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:44.454959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:44.454959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:44.454959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:44.454959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:44.454959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:44.454959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:44.454959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:44.454959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:44.454959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:44.454959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:44.455887 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:54:44.455887 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:54:44.455887 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:54:44.455887 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:54:44.455887 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:54:44.455887 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:54:44.455887 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:44.455887 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:44.459152 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 332/650] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:54:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:44.460532 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f87385-d2c4-4de2-89a3-690a6afe1726 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:44.478106 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:44.521586 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf2b25-8f40-4eb0-8b94-d4730c233812 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:44.526569 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:44.559956 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c02a07-b18e-4ad1-a6d0-859f70bc770d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:44.560577 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c02a07-b18e-4ad1-a6d0-859f70bc770d None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:44.560857 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c02a07-b18e-4ad1-a6d0-859f70bc770d None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:44.561185 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c02a07-b18e-4ad1-a6d0-859f70bc770d None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:44.561939 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c02a07-b18e-4ad1-a6d0-859f70bc770d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:44.562279 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c02a07-b18e-4ad1-a6d0-859f70bc770d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:44.563698 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c02a07-b18e-4ad1-a6d0-859f70bc770d None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:44.579600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 319/651] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:44.579600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c02a07-b18e-4ad1-a6d0-859f70bc770d None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:44.579859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f87385-d2c4-4de2-89a3-690a6afe1726 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:44.618575 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba51a2-6220-4495-a09f-025b77927ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:44.619242 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba51a2-6220-4495-a09f-025b77927ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:44.619524 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba51a2-6220-4495-a09f-025b77927ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:44.619802 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba51a2-6220-4495-a09f-025b77927ab8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:44.631226 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba51a2-6220-4495-a09f-025b77927ab8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:44.632698 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba51a2-6220-4495-a09f-025b77927ab8 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:44.647560 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-2aba51a2-6220-4495-a09f-025b77927ab8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:44.647560 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:44.647560 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:44.647560 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:44.647560 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:44.647560 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:44.647560 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:44.647560 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:44.647560 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:44.647560 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:44.647560 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:44.647560 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:44.647560 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:44.647560 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:44.647560 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:44.647560 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:44.648646 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:44.648646 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:44.648646 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:44.648646 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:44.648646 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:44.648646 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:44.648646 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:44.648646 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:44.648646 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:44.648646 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:44.648646 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:44.648646 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:44.648646 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:44.648646 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:44.648646 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:44.648646 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:44.648646 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:44.649557 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:44.649557 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:44.649557 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:44.649557 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 333/652] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:44.649557 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba51a2-6220-4495-a09f-025b77927ab8 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:44.695121 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c02a07-b18e-4ad1-a6d0-859f70bc770d None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:44.716361 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:44.732345 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b12f5f-f05b-44f5-90f3-210181fe3742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:44.733952 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b12f5f-f05b-44f5-90f3-210181fe3742 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:44.734438 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b12f5f-f05b-44f5-90f3-210181fe3742 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:44.734956 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b12f5f-f05b-44f5-90f3-210181fe3742 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:44.748210 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b12f5f-f05b-44f5-90f3-210181fe3742 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:44.750828 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b12f5f-f05b-44f5-90f3-210181fe3742 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:44.762817 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba51a2-6220-4495-a09f-025b77927ab8 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:44.768132 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-41b12f5f-f05b-44f5-90f3-210181fe3742 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:44.768132 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:44.768132 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:44.768132 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:44.768132 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:44.768132 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:44.768132 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:44.768132 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:44.768132 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:44.768132 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:44.768132 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:44.768132 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:44.768132 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:44.768132 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:44.768132 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:44.768132 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:44.769334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:44.769334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:44.769334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:44.769334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:44.769334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:44.769334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:44.769334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:44.769334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:44.769334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:44.769334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:44.769334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:44.769334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:44.769334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:44.769334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:44.769334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:44.769334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:44.769334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:44.770280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:44.770280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:44.770280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:44.770280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 320/653] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:44.770515 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b12f5f-f05b-44f5-90f3-210181fe3742 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:44.776103 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:44.801120 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b7811-6248-4d7e-8573-afa91bd54479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:44.801726 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b7811-6248-4d7e-8573-afa91bd54479 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:44.802007 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b7811-6248-4d7e-8573-afa91bd54479 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:44.802280 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b7811-6248-4d7e-8573-afa91bd54479 None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:44.802993 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0b7811-6248-4d7e-8573-afa91bd54479 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:44.803299 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0b7811-6248-4d7e-8573-afa91bd54479 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:44.804831 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0b7811-6248-4d7e-8573-afa91bd54479 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:44.817449 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 334/654] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:54:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:44.818238 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b7811-6248-4d7e-8573-afa91bd54479 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:44.872516 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b12f5f-f05b-44f5-90f3-210181fe3742 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:44.908302 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b063ed44-9b2a-49b6-9430-170599462a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:44.908931 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063ed44-9b2a-49b6-9430-170599462a02 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:44.909214 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063ed44-9b2a-49b6-9430-170599462a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:44.909485 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063ed44-9b2a-49b6-9430-170599462a02 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:44.916795 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b063ed44-9b2a-49b6-9430-170599462a02 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:44.918135 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b7811-6248-4d7e-8573-afa91bd54479 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:44.918586 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b063ed44-9b2a-49b6-9430-170599462a02 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:44.930118 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-b063ed44-9b2a-49b6-9430-170599462a02 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 14:54:44.930118 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:44.930118 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:44.930118 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:44.930118 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:44.930118 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:44.930118 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:44.930118 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:44.930118 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:44.930118 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:44.930118 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:44.930118 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:44.930118 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:54:44.930118 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:54:44.930118 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:54:44.930118 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:54:44.931259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:44.931259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:44.931259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:54:44.931259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:44.931259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:54:44.931259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:44.931259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:44.931259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:44.931259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:44.931259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:44.931259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:44.931259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:44.931259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:44.931259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:44.931259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:44.931259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:44.931259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:44.931259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:44.932417 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:44.932417 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:44.932417 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:54:44.932417 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:54:44.932417 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:54:44.932417 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:54:44.932417 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:54:44.932417 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:54:44.932417 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 14:54:44.932417 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:44.932417 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 321/655] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:54:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:44.932417 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af061491-1485-4e06-ab33-86580f1bf0aa None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:44.933479 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af061491-1485-4e06-ab33-86580f1bf0aa None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:44.933479 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af061491-1485-4e06-ab33-86580f1bf0aa None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:44.934568 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 322/656] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:44.935133 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af061491-1485-4e06-ab33-86580f1bf0aa None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:44.960660 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87bf039-b63c-41b1-8eca-a2ee0ff8bfdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:44.961283 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87bf039-b63c-41b1-8eca-a2ee0ff8bfdd None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:44.961562 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87bf039-b63c-41b1-8eca-a2ee0ff8bfdd None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:44.961839 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87bf039-b63c-41b1-8eca-a2ee0ff8bfdd None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:44.962485 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87bf039-b63c-41b1-8eca-a2ee0ff8bfdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:44.962784 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87bf039-b63c-41b1-8eca-a2ee0ff8bfdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:44.964187 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87bf039-b63c-41b1-8eca-a2ee0ff8bfdd None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:44.966943 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:44.976892 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 335/657] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:44.977529 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87bf039-b63c-41b1-8eca-a2ee0ff8bfdd None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:45.007057 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:45.046268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af061491-1485-4e06-ab33-86580f1bf0aa None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:45.078160 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87bf039-b63c-41b1-8eca-a2ee0ff8bfdd None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:45.080680 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926763f7-7ce7-44de-89c2-6dac48438f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:45.081293 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926763f7-7ce7-44de-89c2-6dac48438f65 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:45.081575 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926763f7-7ce7-44de-89c2-6dac48438f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:45.081848 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926763f7-7ce7-44de-89c2-6dac48438f65 None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:45.082489 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926763f7-7ce7-44de-89c2-6dac48438f65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:45.082792 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926763f7-7ce7-44de-89c2-6dac48438f65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:45.084209 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926763f7-7ce7-44de-89c2-6dac48438f65 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:45.096361 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 323/658] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:45.097147 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926763f7-7ce7-44de-89c2-6dac48438f65 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:45.112284 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42bdae-0c4f-4460-a052-25449f8d7b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:45.112893 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42bdae-0c4f-4460-a052-25449f8d7b81 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:45.113170 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42bdae-0c4f-4460-a052-25449f8d7b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:45.113447 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42bdae-0c4f-4460-a052-25449f8d7b81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:45.114166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e42bdae-0c4f-4460-a052-25449f8d7b81 None admin] RBAC: Adding query filter params (role.id=533fa341c4f64cd4b0a7ebb668f9106a, scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176, user.id=fc0ea93084b0482fb2dd6d802987942f) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:45.114468 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e42bdae-0c4f-4460-a052-25449f8d7b81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:45.115943 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e42bdae-0c4f-4460-a052-25449f8d7b81 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:45.122400 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 336/659] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:54:44 2022] GET /identity/v3/role_assignments?user.id=fc0ea93084b0482fb2dd6d802987942f&scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176&role.id=533fa341c4f64cd4b0a7ebb668f9106a => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:45.124061 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c4627-8687-4651-a8d9-aecdfd1fca6f None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:45.124381 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c4627-8687-4651-a8d9-aecdfd1fca6f None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:45.125118 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c4627-8687-4651-a8d9-aecdfd1fca6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:45.147384 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-763c4627-8687-4651-a8d9-aecdfd1fca6f None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:45.148626 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-763c4627-8687-4651-a8d9-aecdfd1fca6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:45.197508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:45.208270 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926763f7-7ce7-44de-89c2-6dac48438f65 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:45.226986 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 337/660] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:45.228113 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c4627-8687-4651-a8d9-aecdfd1fca6f None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:45.257445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:45.266365 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de417c-1b85-4dcf-b139-3ac97dc43ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:45.267570 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de417c-1b85-4dcf-b139-3ac97dc43ae4 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:45.268049 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de417c-1b85-4dcf-b139-3ac97dc43ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:45.268536 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de417c-1b85-4dcf-b139-3ac97dc43ae4 None admin] PATH_INFO: `/v3/users` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:45.269790 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de417c-1b85-4dcf-b139-3ac97dc43ae4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:45.270322 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de417c-1b85-4dcf-b139-3ac97dc43ae4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:45.273036 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de417c-1b85-4dcf-b139-3ac97dc43ae4 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:45.299267 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 324/661] 10.222.0.33 () {56 vars in 1137 bytes} [Sun Sep 11 14:54:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:45.301068 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de417c-1b85-4dcf-b139-3ac97dc43ae4 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:45.351692 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c4627-8687-4651-a8d9-aecdfd1fca6f None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:45.393602 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c2470-969c-4aaf-8bc2-02ca7aa3532c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:45.393602 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c2470-969c-4aaf-8bc2-02ca7aa3532c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:45.393893 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c2470-969c-4aaf-8bc2-02ca7aa3532c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:45.395298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c2470-969c-4aaf-8bc2-02ca7aa3532c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:45.404528 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c2470-969c-4aaf-8bc2-02ca7aa3532c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:45.407023 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c2470-969c-4aaf-8bc2-02ca7aa3532c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:45.419456 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-840c2470-969c-4aaf-8bc2-02ca7aa3532c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 14:54:45.419456 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:45.419456 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:45.419456 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:45.419456 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:45.419456 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:45.419456 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:45.419456 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:45.419456 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:45.419456 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:45.419456 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:45.419456 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:45.419456 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:54:45.419456 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:54:45.419456 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:54:45.419456 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:54:45.420607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:45.420607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:45.420607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:54:45.420607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:45.420607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:54:45.420607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:45.420607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:45.420607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:45.420607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:45.420607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:45.420607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:45.420607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:45.420607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:45.420607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:45.420607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:45.420607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:45.420607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:45.420607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:45.422042 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:45.422042 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:45.422042 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:54:45.422042 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:54:45.422042 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:54:45.422042 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:54:45.422042 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:54:45.422042 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:54:45.422042 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 14:54:45.422042 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:45.422042 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 338/662] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 14:54:45 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:45.422042 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c2470-969c-4aaf-8bc2-02ca7aa3532c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:45.434795 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de417c-1b85-4dcf-b139-3ac97dc43ae4 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:45.476056 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b03589-3d44-47c8-99b4-59d202d4d2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:45.476810 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b03589-3d44-47c8-99b4-59d202d4d2f8 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:45.477152 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b03589-3d44-47c8-99b4-59d202d4d2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:45.477489 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b03589-3d44-47c8-99b4-59d202d4d2f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:45.487475 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b03589-3d44-47c8-99b4-59d202d4d2f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:45.489225 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b03589-3d44-47c8-99b4-59d202d4d2f8 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:45.502952 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-98b03589-3d44-47c8-99b4-59d202d4d2f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:45.502952 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:45.502952 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:45.502952 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:45.502952 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:45.502952 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:45.502952 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:45.502952 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:45.502952 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:45.502952 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:45.502952 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:45.502952 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:45.502952 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:54:45.502952 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:54:45.502952 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:54:45.503993 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:54:45.503993 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:45.503993 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:45.503993 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:45.503993 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:45.503993 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:45.503993 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:45.503993 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:45.503993 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:45.503993 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:45.503993 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:45.503993 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:45.503993 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:45.503993 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:45.503993 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:45.503993 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:45.503993 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:45.504902 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:54:45.504902 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:54:45.504902 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:54:45.504902 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:54:45.504902 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:54:45.504902 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:54:45.504902 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:45.504902 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:45.504902 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 325/663] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:54:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:45.505388 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b03589-3d44-47c8-99b4-59d202d4d2f8 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:45.517658 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:45.526723 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:45.539113 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c2470-969c-4aaf-8bc2-02ca7aa3532c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:45.576091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd76b30-5a80-4f18-8944-a26ba207a862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:45.576624 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd76b30-5a80-4f18-8944-a26ba207a862 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:45.577459 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd76b30-5a80-4f18-8944-a26ba207a862 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:45.577734 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd76b30-5a80-4f18-8944-a26ba207a862 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:45.618313 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b03589-3d44-47c8-99b4-59d202d4d2f8 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:45.648790 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd76b30-5a80-4f18-8944-a26ba207a862 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:45.650251 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd76b30-5a80-4f18-8944-a26ba207a862 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:45.656754 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769e9b8-4b32-4bb5-a6e0-5f6dfa978ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:45.657372 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769e9b8-4b32-4bb5-a6e0-5f6dfa978ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:45.657655 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769e9b8-4b32-4bb5-a6e0-5f6dfa978ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:45.657930 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769e9b8-4b32-4bb5-a6e0-5f6dfa978ef3 None admin] PATH_INFO: `/v3/users` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:45.659108 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c769e9b8-4b32-4bb5-a6e0-5f6dfa978ef3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:45.659429 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c769e9b8-4b32-4bb5-a6e0-5f6dfa978ef3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:45.661262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c769e9b8-4b32-4bb5-a6e0-5f6dfa978ef3 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:45.685433 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 326/664] 10.222.0.33 () {56 vars in 1139 bytes} [Sun Sep 11 14:54:45 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:45.685433 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769e9b8-4b32-4bb5-a6e0-5f6dfa978ef3 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:45.750534 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:45.758169 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:45.774309 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 339/665] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:54:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:54:45.774309 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd76b30-5a80-4f18-8944-a26ba207a862 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:45.801178 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769e9b8-4b32-4bb5-a6e0-5f6dfa978ef3 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:45.838537 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53401e34-12a3-4ab2-b412-adc18a1ceb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:45.839099 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53401e34-12a3-4ab2-b412-adc18a1ceb25 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:45.839383 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53401e34-12a3-4ab2-b412-adc18a1ceb25 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:45.839659 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53401e34-12a3-4ab2-b412-adc18a1ceb25 None admin] PATH_INFO: `/v3/projects` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:45.840348 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53401e34-12a3-4ab2-b412-adc18a1ceb25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:45.840821 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53401e34-12a3-4ab2-b412-adc18a1ceb25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:45.842832 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53401e34-12a3-4ab2-b412-adc18a1ceb25 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:45.854872 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 327/666] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:54:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:45.855577 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53401e34-12a3-4ab2-b412-adc18a1ceb25 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:45.880109 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd76b30-5a80-4f18-8944-a26ba207a862 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:45.917303 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de059c2-b863-46b8-8a4a-6fb22a607e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:45.917906 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de059c2-b863-46b8-8a4a-6fb22a607e26 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:45.918191 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de059c2-b863-46b8-8a4a-6fb22a607e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:45.918747 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de059c2-b863-46b8-8a4a-6fb22a607e26 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:45.930595 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de059c2-b863-46b8-8a4a-6fb22a607e26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:45.935531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-8de059c2-b863-46b8-8a4a-6fb22a607e26 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:54:45.935531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:45.935531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:45.935531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:45.935531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:45.935531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:45.935531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:45.935531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:45.935531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:45.935531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:45.935531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:45.935531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:45.935531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:45.935531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:45.935531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:45.935531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:45.936514 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:45.936514 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:45.936514 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:45.936514 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:45.936514 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:45.936514 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:45.936514 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:45.936514 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:45.936514 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:45.936514 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:45.936514 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:45.936514 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:45.936514 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:45.936514 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:45.936514 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:45.936514 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:45.939900 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:45.939900 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:45.939900 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:45.939900 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:45.939900 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:45.939900 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:45.939900 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:45.939900 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:45.939900 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:45.939900 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:45.939900 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:54:45.939900 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:45.939900 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 340/667] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 14:54:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:45.939900 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de059c2-b863-46b8-8a4a-6fb22a607e26 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:45.971087 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53401e34-12a3-4ab2-b412-adc18a1ceb25 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:45.996040 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:46.002399 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:46.011759 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fe5b82-46a1-4466-8de5-b120e5faad5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:46.012376 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe5b82-46a1-4466-8de5-b120e5faad5e None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:46.012660 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe5b82-46a1-4466-8de5-b120e5faad5e None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:46.012933 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe5b82-46a1-4466-8de5-b120e5faad5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:46.058480 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de059c2-b863-46b8-8a4a-6fb22a607e26 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:46.086991 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fe5b82-46a1-4466-8de5-b120e5faad5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:46.088445 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fe5b82-46a1-4466-8de5-b120e5faad5e None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:46.095142 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f9aff-1450-45a8-b39e-5451bad101ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:46.095771 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f9aff-1450-45a8-b39e-5451bad101ee None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:46.096053 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f9aff-1450-45a8-b39e-5451bad101ee None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:46.096326 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f9aff-1450-45a8-b39e-5451bad101ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:46.107347 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671f9aff-1450-45a8-b39e-5451bad101ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:46.108974 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671f9aff-1450-45a8-b39e-5451bad101ee None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:46.126874 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-671f9aff-1450-45a8-b39e-5451bad101ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:46.126874 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:46.126874 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:46.126874 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:46.126874 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:46.126874 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:46.126874 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:46.126874 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:46.126874 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:46.126874 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:46.126874 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:46.126874 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:46.126874 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:46.126874 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:46.126874 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:46.126874 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:46.127916 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:46.127916 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:46.127916 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:46.127916 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:46.127916 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:46.127916 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:46.127916 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:46.127916 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:46.127916 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:46.127916 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:46.127916 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:46.127916 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:46.127916 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:46.127916 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:46.127916 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:46.127916 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:46.127916 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:46.128814 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:46.128814 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:46.128814 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:46.132877 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 341/668] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:46.133645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f9aff-1450-45a8-b39e-5451bad101ee None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:46.204345 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 328/669] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:54:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:54:46.205362 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fe5b82-46a1-4466-8de5-b120e5faad5e None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:46.235993 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:46.236212 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:46.243901 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f9aff-1450-45a8-b39e-5451bad101ee None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:46.281676 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7bd5a-3874-4c22-a798-06c92629a074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:46.282307 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7bd5a-3874-4c22-a798-06c92629a074 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:46.282583 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7bd5a-3874-4c22-a798-06c92629a074 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:46.282888 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7bd5a-3874-4c22-a798-06c92629a074 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:46.283605 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c7bd5a-3874-4c22-a798-06c92629a074 None admin] RBAC: Adding query filter params (role.id=533fa341c4f64cd4b0a7ebb668f9106a, scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176, user.id=cd9c8b53871e415facea40c6694f4171) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:46.283909 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c7bd5a-3874-4c22-a798-06c92629a074 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:46.285351 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c7bd5a-3874-4c22-a798-06c92629a074 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:46.291703 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 342/670] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:54:46 2022] GET /identity/v3/role_assignments?user.id=cd9c8b53871e415facea40c6694f4171&scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176&role.id=533fa341c4f64cd4b0a7ebb668f9106a => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:46.292941 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7bd5a-3874-4c22-a798-06c92629a074 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:46.319318 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fe5b82-46a1-4466-8de5-b120e5faad5e None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:46.365466 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632977a5-bf07-45f7-bdc5-1bc3f773b5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:46.366448 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632977a5-bf07-45f7-bdc5-1bc3f773b5df None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:46.366940 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632977a5-bf07-45f7-bdc5-1bc3f773b5df None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:46.367418 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632977a5-bf07-45f7-bdc5-1bc3f773b5df None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:46.368281 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632977a5-bf07-45f7-bdc5-1bc3f773b5df None admin] RBAC: Adding query filter params (name=service) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:46.368765 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632977a5-bf07-45f7-bdc5-1bc3f773b5df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:46.370382 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632977a5-bf07-45f7-bdc5-1bc3f773b5df None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:46.380372 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 329/671] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 14:54:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:46.381676 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632977a5-bf07-45f7-bdc5-1bc3f773b5df None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:46.456358 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7bd5a-3874-4c22-a798-06c92629a074 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:46.492388 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:46.501816 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a1cf6-ef3f-4f26-9aed-41360ded0ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:46.502571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a1cf6-ef3f-4f26-9aed-41360ded0ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:46.502879 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a1cf6-ef3f-4f26-9aed-41360ded0ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:46.503157 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a1cf6-ef3f-4f26-9aed-41360ded0ad8 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:46.503812 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a1cf6-ef3f-4f26-9aed-41360ded0ad8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:46.504114 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a1cf6-ef3f-4f26-9aed-41360ded0ad8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:46.506463 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a1cf6-ef3f-4f26-9aed-41360ded0ad8 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:46.511161 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632977a5-bf07-45f7-bdc5-1bc3f773b5df None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:46.525597 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 343/672] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:46.526529 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a1cf6-ef3f-4f26-9aed-41360ded0ad8 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:46.554627 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:46.557594 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4fcf21-91ca-44ae-9615-56376d624adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:46.558572 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fcf21-91ca-44ae-9615-56376d624adc None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:46.558920 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fcf21-91ca-44ae-9615-56376d624adc None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:46.559662 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fcf21-91ca-44ae-9615-56376d624adc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:46.575449 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4fcf21-91ca-44ae-9615-56376d624adc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:46.576425 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4fcf21-91ca-44ae-9615-56376d624adc None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:46.596107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-0c4fcf21-91ca-44ae-9615-56376d624adc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:46.596107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:46.596107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:46.596107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:46.596107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:46.596107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:46.596107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:46.596107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:46.596107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:46.596107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:46.596107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:46.596107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:46.596107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:46.596107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:46.596107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:46.596107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:46.597191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:46.597191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:46.597191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:46.597191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:46.597191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:46.597191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:46.597191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:46.597191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:46.597191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:46.597191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:46.597191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:46.597191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:46.597191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:46.597191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:46.597191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:46.597191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:46.597191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:46.598108 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:46.598108 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:46.598108 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:46.598108 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 330/673] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:46.598623 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4fcf21-91ca-44ae-9615-56376d624adc None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:46.653640 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a1cf6-ef3f-4f26-9aed-41360ded0ad8 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:46.702273 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf6601-8c32-406f-85ca-6a86225c418f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:46.703578 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf6601-8c32-406f-85ca-6a86225c418f None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:46.703967 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf6601-8c32-406f-85ca-6a86225c418f None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:46.704342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf6601-8c32-406f-85ca-6a86225c418f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:46.712769 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4fcf21-91ca-44ae-9615-56376d624adc None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:46.719130 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf6601-8c32-406f-85ca-6a86225c418f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:46.720851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf6601-8c32-406f-85ca-6a86225c418f None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:46.741674 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-7ecf6601-8c32-406f-85ca-6a86225c418f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:46.741674 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:46.741674 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:46.741674 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:46.741674 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:46.741674 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:46.741674 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:46.741674 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:46.741674 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:46.741674 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:46.741674 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:46.741674 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:46.741674 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:46.741674 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:46.741674 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:46.741674 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:46.742716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:46.742716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:46.742716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:46.742716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:46.742716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:46.742716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:46.742716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:46.742716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:46.742716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:46.742716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:46.742716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:46.742716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:46.742716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:46.742716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:46.742716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:46.742716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:46.742716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:46.743664 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:46.743664 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:46.743664 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:46.743664 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 344/674] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:46.745134 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf6601-8c32-406f-85ca-6a86225c418f None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:46.753677 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36ecbf-0c65-410b-86ff-6c517effd5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:46.754319 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36ecbf-0c65-410b-86ff-6c517effd5ad None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:46.754601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36ecbf-0c65-410b-86ff-6c517effd5ad None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:46.755175 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36ecbf-0c65-410b-86ff-6c517effd5ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:46.763062 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e36ecbf-0c65-410b-86ff-6c517effd5ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:46.764866 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e36ecbf-0c65-410b-86ff-6c517effd5ad None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:46.777600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-6e36ecbf-0c65-410b-86ff-6c517effd5ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:46.777600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:46.777600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:46.777600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:46.777600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:46.777600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:46.777600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:46.777600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:46.777600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:46.777600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:46.777600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:46.777600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:46.777600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:54:46.777600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:54:46.777600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:54:46.778511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:54:46.778511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:46.778511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:46.778511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:46.778511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:46.778511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:46.778511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:46.778511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:46.778511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:46.778511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:46.778511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:46.778511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:46.778511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:46.778511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:46.778511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:46.778511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:46.778511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:46.779455 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:54:46.779455 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:54:46.779455 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:54:46.779455 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:54:46.779455 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:54:46.779455 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:54:46.779455 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:46.779455 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:46.783061 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 331/675] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:54:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:46.784047 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36ecbf-0c65-410b-86ff-6c517effd5ad None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:46.815166 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:46.829978 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:46.870235 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf6601-8c32-406f-85ca-6a86225c418f None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:46.909958 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b61fd1-bcf8-4ae7-9b8f-694e3b0abbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:46.910164 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36ecbf-0c65-410b-86ff-6c517effd5ad None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:46.911071 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b61fd1-bcf8-4ae7-9b8f-694e3b0abbfd None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:46.911844 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b61fd1-bcf8-4ae7-9b8f-694e3b0abbfd None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:46.913203 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b61fd1-bcf8-4ae7-9b8f-694e3b0abbfd None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:46.914070 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b61fd1-bcf8-4ae7-9b8f-694e3b0abbfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:46.914580 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b61fd1-bcf8-4ae7-9b8f-694e3b0abbfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:46.916219 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b61fd1-bcf8-4ae7-9b8f-694e3b0abbfd None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:46.936200 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 345/676] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:46.937256 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b61fd1-bcf8-4ae7-9b8f-694e3b0abbfd None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:46.953606 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb237f-8384-4fe3-8eed-1094c61cb825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:46.954228 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb237f-8384-4fe3-8eed-1094c61cb825 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:46.954513 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb237f-8384-4fe3-8eed-1094c61cb825 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:46.954789 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb237f-8384-4fe3-8eed-1094c61cb825 None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:46.955470 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cb237f-8384-4fe3-8eed-1094c61cb825 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:46.955773 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cb237f-8384-4fe3-8eed-1094c61cb825 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:46.957184 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cb237f-8384-4fe3-8eed-1094c61cb825 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:46.977603 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 332/677] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:46.977603 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb237f-8384-4fe3-8eed-1094c61cb825 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:47.071034 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:47.080139 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b61fd1-bcf8-4ae7-9b8f-694e3b0abbfd None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:47.113024 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:47.120031 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1adace-73ba-481f-ab2d-04689b1c02ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:47.120858 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1adace-73ba-481f-ab2d-04689b1c02ab None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:47.121241 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1adace-73ba-481f-ab2d-04689b1c02ab None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:47.121614 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1adace-73ba-481f-ab2d-04689b1c02ab None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:47.122411 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1adace-73ba-481f-ab2d-04689b1c02ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:47.122831 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1adace-73ba-481f-ab2d-04689b1c02ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:47.124515 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1adace-73ba-481f-ab2d-04689b1c02ab None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:47.125157 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb237f-8384-4fe3-8eed-1094c61cb825 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:47.142972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 346/678] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:54:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:47.144120 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1adace-73ba-481f-ab2d-04689b1c02ab None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:47.167077 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb087955-0791-4a62-95e2-b9a1fa2fd812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:47.167957 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb087955-0791-4a62-95e2-b9a1fa2fd812 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:47.168380 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb087955-0791-4a62-95e2-b9a1fa2fd812 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:47.168790 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb087955-0791-4a62-95e2-b9a1fa2fd812 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:47.177443 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb087955-0791-4a62-95e2-b9a1fa2fd812 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:47.179381 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb087955-0791-4a62-95e2-b9a1fa2fd812 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:47.192675 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-fb087955-0791-4a62-95e2-b9a1fa2fd812 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:47.192675 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:47.192675 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:47.192675 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:47.192675 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:47.192675 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:47.192675 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:47.192675 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:47.192675 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:47.192675 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:47.192675 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:47.192675 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:47.192675 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:54:47.192675 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:54:47.192675 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:54:47.193600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:54:47.193600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:47.193600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:47.193600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:47.193600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:47.193600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:47.193600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:47.193600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:47.193600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:47.193600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:47.193600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:47.193600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:47.193600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:47.193600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:47.193600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:47.193600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:47.193600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:47.194501 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:54:47.194501 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:54:47.194501 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:54:47.194501 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:54:47.194501 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:54:47.194501 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:54:47.194501 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:47.194501 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:47.203356 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 333/679] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:54:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:47.203356 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb087955-0791-4a62-95e2-b9a1fa2fd812 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:47.271444 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1adace-73ba-481f-ab2d-04689b1c02ab None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:47.322259 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf522bc8-c772-4e68-aaa2-24c5e02d9449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:47.323320 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf522bc8-c772-4e68-aaa2-24c5e02d9449 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:47.323430 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb087955-0791-4a62-95e2-b9a1fa2fd812 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:47.323900 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf522bc8-c772-4e68-aaa2-24c5e02d9449 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:47.324394 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf522bc8-c772-4e68-aaa2-24c5e02d9449 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:47.335833 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf522bc8-c772-4e68-aaa2-24c5e02d9449 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:47.338374 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf522bc8-c772-4e68-aaa2-24c5e02d9449 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:47.351844 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:47.355200 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-cf522bc8-c772-4e68-aaa2-24c5e02d9449 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 14:54:47.355200 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:47.355200 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:47.355200 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:47.355200 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:47.355200 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:47.355200 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:47.355200 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:47.355200 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:47.355200 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:47.355200 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:47.355200 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:47.355200 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:54:47.355200 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:54:47.355200 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:54:47.355200 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:54:47.356132 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:47.356132 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:47.356132 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:54:47.356132 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:47.356132 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:54:47.356132 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:47.356132 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:47.356132 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:47.356132 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:47.356132 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:47.356132 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:47.356132 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:47.356132 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:47.356132 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:47.356132 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:47.356132 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:47.356132 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:47.356132 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:47.357187 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:47.357187 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:47.357187 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:54:47.357187 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:54:47.357187 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:54:47.357187 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:54:47.357187 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:54:47.357187 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:54:47.357187 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 14:54:47.357187 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:47.359344 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 347/680] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:54:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:47.361210 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf522bc8-c772-4e68-aaa2-24c5e02d9449 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:47.373144 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d4a227-e2af-4498-8a0e-c3512ce84095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:47.374588 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d4a227-e2af-4498-8a0e-c3512ce84095 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:47.375316 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d4a227-e2af-4498-8a0e-c3512ce84095 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:47.375733 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d4a227-e2af-4498-8a0e-c3512ce84095 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:47.377295 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d4a227-e2af-4498-8a0e-c3512ce84095 None admin] RBAC: Adding query filter params (role.id=533fa341c4f64cd4b0a7ebb668f9106a, scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176, user.id=c793eafcdba249a5af3ab948e9774367) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:47.377791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d4a227-e2af-4498-8a0e-c3512ce84095 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:47.380229 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d4a227-e2af-4498-8a0e-c3512ce84095 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:47.388013 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 334/681] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:54:47 2022] GET /identity/v3/role_assignments?user.id=c793eafcdba249a5af3ab948e9774367&scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176&role.id=533fa341c4f64cd4b0a7ebb668f9106a => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:47.389090 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d4a227-e2af-4498-8a0e-c3512ce84095 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:47.454040 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:47.511253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf522bc8-c772-4e68-aaa2-24c5e02d9449 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:47.524747 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d4a227-e2af-4498-8a0e-c3512ce84095 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:47.550115 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe98f85-1870-4a69-9261-d7bd267e6380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:47.550909 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe98f85-1870-4a69-9261-d7bd267e6380 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:47.551264 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe98f85-1870-4a69-9261-d7bd267e6380 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:47.551633 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe98f85-1870-4a69-9261-d7bd267e6380 None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:47.552715 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe98f85-1870-4a69-9261-d7bd267e6380 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:47.553093 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe98f85-1870-4a69-9261-d7bd267e6380 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:47.555473 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe98f85-1870-4a69-9261-d7bd267e6380 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:47.567406 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dbfbc5-e6b3-4c7f-8498-accef972e2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:47.568039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbfbc5-e6b3-4c7f-8498-accef972e2b6 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:47.568319 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbfbc5-e6b3-4c7f-8498-accef972e2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:47.568593 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbfbc5-e6b3-4c7f-8498-accef972e2b6 None admin] PATH_INFO: `/v3/users` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:47.569336 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dbfbc5-e6b3-4c7f-8498-accef972e2b6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:47.569641 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dbfbc5-e6b3-4c7f-8498-accef972e2b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:47.571131 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dbfbc5-e6b3-4c7f-8498-accef972e2b6 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:47.571373 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 348/682] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:54:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:47.572638 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234a55a-af51-4d03-abf6-0206ff25f726 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:47.572924 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234a55a-af51-4d03-abf6-0206ff25f726 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:47.573199 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234a55a-af51-4d03-abf6-0206ff25f726 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:47.574403 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 349/683] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:47.581220 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6f4b4-78cf-439f-8fec-7a79977e716e None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:47.581496 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6f4b4-78cf-439f-8fec-7a79977e716e None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:47.581769 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6f4b4-78cf-439f-8fec-7a79977e716e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:47.597894 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 335/684] 10.222.0.33 () {56 vars in 1135 bytes} [Sun Sep 11 14:54:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:47.599187 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dbfbc5-e6b3-4c7f-8498-accef972e2b6 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:47.600721 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-a0b6f4b4-78cf-439f-8fec-7a79977e716e None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:47.602148 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a0b6f4b4-78cf-439f-8fec-7a79977e716e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:47.658505 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:47.694831 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 350/685] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:47.694831 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6f4b4-78cf-439f-8fec-7a79977e716e None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:47.725846 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dbfbc5-e6b3-4c7f-8498-accef972e2b6 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:47.762698 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c282f93d-0941-4b5b-ba09-b47546ea0e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:47.763343 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282f93d-0941-4b5b-ba09-b47546ea0e36 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:47.763881 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282f93d-0941-4b5b-ba09-b47546ea0e36 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:47.763881 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282f93d-0941-4b5b-ba09-b47546ea0e36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:47.765124 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c282f93d-0941-4b5b-ba09-b47546ea0e36 None admin] RBAC: Adding query filter params (role.id=533fa341c4f64cd4b0a7ebb668f9106a, scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176, user.id=df6eb236e80b4395a7aa77020b2f3a4f) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:47.765434 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c282f93d-0941-4b5b-ba09-b47546ea0e36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:47.766303 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:47.766908 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c282f93d-0941-4b5b-ba09-b47546ea0e36 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:47.773850 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 336/686] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:54:47 2022] GET /identity/v3/role_assignments?user.id=df6eb236e80b4395a7aa77020b2f3a4f&scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176&role.id=533fa341c4f64cd4b0a7ebb668f9106a => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:47.868022 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6f4b4-78cf-439f-8fec-7a79977e716e None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:47.924933 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794aae4-241b-4b5f-b11f-fca8998cf6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:47.925932 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794aae4-241b-4b5f-b11f-fca8998cf6a7 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:47.926357 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794aae4-241b-4b5f-b11f-fca8998cf6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:47.926770 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794aae4-241b-4b5f-b11f-fca8998cf6a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:48.015524 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2794aae4-241b-4b5f-b11f-fca8998cf6a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:48.017039 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2794aae4-241b-4b5f-b11f-fca8998cf6a7 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:48.061661 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:48.108440 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c282f93d-0941-4b5b-ba09-b47546ea0e36 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:48.123241 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 351/687] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:54:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:54:48.129373 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794aae4-241b-4b5f-b11f-fca8998cf6a7 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:48.244018 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c282f93d-0941-4b5b-ba09-b47546ea0e36 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:48.253319 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794aae4-241b-4b5f-b11f-fca8998cf6a7 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:48.257637 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:48.291685 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc60e5-addc-4aef-b0af-05be360f2eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:48.292320 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc60e5-addc-4aef-b0af-05be360f2eb8 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:48.292597 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc60e5-addc-4aef-b0af-05be360f2eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:48.292863 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc60e5-addc-4aef-b0af-05be360f2eb8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:48.293406 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5887d261-27bc-4a19-94bb-03f5987a6ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:48.294136 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887d261-27bc-4a19-94bb-03f5987a6ec5 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:48.294417 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887d261-27bc-4a19-94bb-03f5987a6ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:48.294691 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887d261-27bc-4a19-94bb-03f5987a6ec5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:48.305837 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5887d261-27bc-4a19-94bb-03f5987a6ec5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:48.306565 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbc60e5-addc-4aef-b0af-05be360f2eb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:48.307353 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5887d261-27bc-4a19-94bb-03f5987a6ec5 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:48.309406 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:48.312790 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-bfbc60e5-addc-4aef-b0af-05be360f2eb8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:54:48.312790 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:48.312790 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:48.312790 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:48.312790 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:48.312790 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:48.312790 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:48.312790 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:48.312790 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:48.312790 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:48.312790 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:48.312790 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:48.312790 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:48.312790 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:48.312790 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:48.312790 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:48.313899 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:48.313899 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:48.313899 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:48.313899 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:48.313899 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:48.313899 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:48.313899 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:48.313899 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:48.313899 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:48.313899 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:48.313899 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:48.313899 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:48.313899 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:48.313899 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:48.313899 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:48.313899 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:48.314744 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:48.314744 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:48.314744 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:48.314744 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:48.314744 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:48.314744 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:48.314744 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:48.314744 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:48.314744 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:48.314744 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:48.314744 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:54:48.314744 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:48.314744 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 337/688] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:54:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:54:48.322887 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc60e5-addc-4aef-b0af-05be360f2eb8 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:48.330732 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-5887d261-27bc-4a19-94bb-03f5987a6ec5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:48.330732 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:48.330732 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:48.330732 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:48.330732 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:48.330732 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:48.330732 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:48.330732 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:48.330732 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:48.330732 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:48.330732 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:48.330732 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:48.330732 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:48.330732 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:48.330732 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:48.330732 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:48.332693 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:48.332693 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:48.332693 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:48.332693 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:48.332693 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:48.332693 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:48.332693 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:48.332693 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:48.332693 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:48.332693 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:48.332693 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:48.332693 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:48.332693 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:48.332693 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:48.332693 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:48.332693 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:48.332693 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:48.333904 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:48.333904 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:48.333904 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:48.333904 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 352/689] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:48.341803 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5887d261-27bc-4a19-94bb-03f5987a6ec5 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:48.435092 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc60e5-addc-4aef-b0af-05be360f2eb8 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:48.473462 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3935b86-8ef5-48de-8443-4c635d70a45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:48.474075 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3935b86-8ef5-48de-8443-4c635d70a45b None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:48.474349 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3935b86-8ef5-48de-8443-4c635d70a45b None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:48.474618 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3935b86-8ef5-48de-8443-4c635d70a45b None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:48.475313 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3935b86-8ef5-48de-8443-4c635d70a45b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:48.475611 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3935b86-8ef5-48de-8443-4c635d70a45b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:48.477074 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3935b86-8ef5-48de-8443-4c635d70a45b None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:48.488269 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 338/690] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:54:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:48.498317 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3935b86-8ef5-48de-8443-4c635d70a45b None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:48.551690 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5887d261-27bc-4a19-94bb-03f5987a6ec5 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:48.557655 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:48.620428 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c50fd-2df5-4ba2-a2d9-501ea569829c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:48.621057 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c50fd-2df5-4ba2-a2d9-501ea569829c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:48.621339 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c50fd-2df5-4ba2-a2d9-501ea569829c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:48.621616 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c50fd-2df5-4ba2-a2d9-501ea569829c None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:48.622263 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585c50fd-2df5-4ba2-a2d9-501ea569829c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:48.622563 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585c50fd-2df5-4ba2-a2d9-501ea569829c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:48.624063 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585c50fd-2df5-4ba2-a2d9-501ea569829c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:48.636858 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 353/691] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:48.642816 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c50fd-2df5-4ba2-a2d9-501ea569829c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:48.663427 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3935b86-8ef5-48de-8443-4c635d70a45b None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:48.700672 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:48.703865 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563082b2-5e16-4d66-974e-4be61c415413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:48.705063 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563082b2-5e16-4d66-974e-4be61c415413 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:48.705346 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563082b2-5e16-4d66-974e-4be61c415413 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:48.705769 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563082b2-5e16-4d66-974e-4be61c415413 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:48.717865 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563082b2-5e16-4d66-974e-4be61c415413 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:48.719370 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563082b2-5e16-4d66-974e-4be61c415413 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:48.735847 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-563082b2-5e16-4d66-974e-4be61c415413 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:48.735847 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:48.735847 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:48.735847 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:48.735847 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:48.735847 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:48.735847 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:48.735847 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:48.735847 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:48.735847 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:48.735847 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:48.735847 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:48.735847 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:48.735847 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:48.735847 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:48.735847 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:48.737801 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:48.737801 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:48.737801 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:48.737801 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:48.737801 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:48.737801 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:48.737801 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:48.737801 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:48.737801 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:48.737801 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:48.737801 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:48.737801 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:48.737801 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:48.737801 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:48.737801 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:48.737801 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:48.737801 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:48.739704 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:48.739704 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:48.739704 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:48.739704 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 339/692] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:54:48.750921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563082b2-5e16-4d66-974e-4be61c415413 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:48.752202 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c50fd-2df5-4ba2-a2d9-501ea569829c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:48.790797 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c59fa-e363-42ad-8eca-7d762227b76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:48.791561 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c59fa-e363-42ad-8eca-7d762227b76a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:48.791908 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c59fa-e363-42ad-8eca-7d762227b76a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:48.792320 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c59fa-e363-42ad-8eca-7d762227b76a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:48.799844 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583c59fa-e363-42ad-8eca-7d762227b76a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:48.801388 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583c59fa-e363-42ad-8eca-7d762227b76a None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:48.813716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-583c59fa-e363-42ad-8eca-7d762227b76a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:48.813716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:48.813716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:48.813716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:48.813716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:48.813716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:48.813716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:48.813716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:48.813716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:48.813716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:48.813716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:48.813716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:48.813716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:54:48.813716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:54:48.813716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:54:48.815546 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:54:48.815546 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:48.815546 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:48.815546 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:48.815546 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:48.815546 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:48.815546 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:48.815546 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:48.815546 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:48.815546 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:48.815546 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:48.815546 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:48.815546 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:48.815546 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:48.815546 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:48.815546 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:48.815546 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:48.817700 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:54:48.817700 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:54:48.817700 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:54:48.817700 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:54:48.817700 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:54:48.817700 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:54:48.817700 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:48.817700 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:48.817700 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 354/693] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:54:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:54:48.822858 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c59fa-e363-42ad-8eca-7d762227b76a None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:48.839807 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:48.862463 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563082b2-5e16-4d66-974e-4be61c415413 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:48.903012 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e39c04-3a97-44fb-94c3-1c1aaf72f0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:48.903630 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e39c04-3a97-44fb-94c3-1c1aaf72f0b1 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:48.903912 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e39c04-3a97-44fb-94c3-1c1aaf72f0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:48.904189 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e39c04-3a97-44fb-94c3-1c1aaf72f0b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:48.905086 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e39c04-3a97-44fb-94c3-1c1aaf72f0b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:48.905518 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e39c04-3a97-44fb-94c3-1c1aaf72f0b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:48.907153 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e39c04-3a97-44fb-94c3-1c1aaf72f0b1 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:48.921014 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 340/694] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:48.926572 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e39c04-3a97-44fb-94c3-1c1aaf72f0b1 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:48.930833 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c59fa-e363-42ad-8eca-7d762227b76a None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:48.946049 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:48.967987 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e2629-05f4-4881-9526-e7bda61973cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:48.968880 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e2629-05f4-4881-9526-e7bda61973cb None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:48.969259 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e2629-05f4-4881-9526-e7bda61973cb None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:48.969632 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e2629-05f4-4881-9526-e7bda61973cb None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:48.970427 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47e2629-05f4-4881-9526-e7bda61973cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:48.970865 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47e2629-05f4-4881-9526-e7bda61973cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:48.977699 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47e2629-05f4-4881-9526-e7bda61973cb None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:48.985627 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 355/695] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:54:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:48.996509 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e2629-05f4-4881-9526-e7bda61973cb None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:49.041461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e39c04-3a97-44fb-94c3-1c1aaf72f0b1 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:49.080651 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb9886-7e66-4909-bad5-b8835856a3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:49.081340 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb9886-7e66-4909-bad5-b8835856a3cc None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:49.081621 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb9886-7e66-4909-bad5-b8835856a3cc None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:49.081939 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb9886-7e66-4909-bad5-b8835856a3cc None admin] PATH_INFO: `/v3/users/nova` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:49.089603 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedb9886-7e66-4909-bad5-b8835856a3cc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:49.091098 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedb9886-7e66-4909-bad5-b8835856a3cc None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:49.100249 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e2629-05f4-4881-9526-e7bda61973cb None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:49.103755 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-bedb9886-7e66-4909-bad5-b8835856a3cc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 14:54:49.103755 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:49.103755 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:49.103755 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:49.103755 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:49.103755 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:49.103755 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:49.103755 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:49.103755 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:49.103755 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:49.103755 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:49.103755 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:49.103755 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:54:49.103755 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:54:49.103755 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:54:49.103755 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:54:49.104691 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:49.104691 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:49.104691 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:54:49.104691 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:49.104691 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:54:49.104691 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:49.104691 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:49.104691 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:49.104691 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:49.104691 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:49.104691 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:49.104691 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:49.104691 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:49.104691 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:49.104691 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:49.104691 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:49.104691 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:49.104691 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:49.105638 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:49.105638 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:49.105638 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:54:49.105638 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:54:49.105638 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:54:49.105638 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:54:49.105638 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:54:49.105638 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:54:49.105638 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 14:54:49.105638 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:49.110157 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 341/696] 10.222.0.33 () {56 vars in 1093 bytes} [Sun Sep 11 14:54:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:54:49.111439 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e87ab-c697-45d0-a502-6fd1be8d7ba8 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:49.111722 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e87ab-c697-45d0-a502-6fd1be8d7ba8 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:49.111998 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e87ab-c697-45d0-a502-6fd1be8d7ba8 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:49.113682 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 342/697] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:49.114273 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e87ab-c697-45d0-a502-6fd1be8d7ba8 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:49.117126 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:49.142114 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1c6d1-ac76-40f6-b937-2af9dc9d01ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:49.143144 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1c6d1-ac76-40f6-b937-2af9dc9d01ff None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:49.143657 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1c6d1-ac76-40f6-b937-2af9dc9d01ff None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:49.144161 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1c6d1-ac76-40f6-b937-2af9dc9d01ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:49.145087 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf1c6d1-ac76-40f6-b937-2af9dc9d01ff None admin] RBAC: Adding query filter params (role.id=533fa341c4f64cd4b0a7ebb668f9106a, scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176, user.id=698f1293294d45b89e04d85649f454d0) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:49.145598 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf1c6d1-ac76-40f6-b937-2af9dc9d01ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:49.147318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf1c6d1-ac76-40f6-b937-2af9dc9d01ff None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:49.158846 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 356/698] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:54:48 2022] GET /identity/v3/role_assignments?user.id=698f1293294d45b89e04d85649f454d0&scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176&role.id=533fa341c4f64cd4b0a7ebb668f9106a => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:49.160738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22db52-5b76-4c18-9410-2aec2707724c None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:49.161253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22db52-5b76-4c18-9410-2aec2707724c None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:49.161728 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22db52-5b76-4c18-9410-2aec2707724c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:49.199913 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-2c22db52-5b76-4c18-9410-2aec2707724c None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:49.201537 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-2c22db52-5b76-4c18-9410-2aec2707724c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:49.235927 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e87ab-c697-45d0-a502-6fd1be8d7ba8 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:49.275185 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:49.275506 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6d8e67-4aa8-41d4-882c-78998fedac0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:49.276157 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d8e67-4aa8-41d4-882c-78998fedac0d None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:49.276435 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d8e67-4aa8-41d4-882c-78998fedac0d None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:49.276788 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d8e67-4aa8-41d4-882c-78998fedac0d None admin] PATH_INFO: `/v3/users` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:49.277617 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6d8e67-4aa8-41d4-882c-78998fedac0d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:49.277929 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6d8e67-4aa8-41d4-882c-78998fedac0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:49.279449 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6d8e67-4aa8-41d4-882c-78998fedac0d None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:49.302026 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 343/699] 10.222.0.33 () {56 vars in 1133 bytes} [Sun Sep 11 14:54:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:49.307538 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6d8e67-4aa8-41d4-882c-78998fedac0d None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:49.357256 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 357/700] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:49.376697 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:49.458192 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6d8e67-4aa8-41d4-882c-78998fedac0d None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:49.493868 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f3181-0c71-4f85-830e-c45a62d3c730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:49.494487 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f3181-0c71-4f85-830e-c45a62d3c730 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:49.494768 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f3181-0c71-4f85-830e-c45a62d3c730 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:49.495074 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f3181-0c71-4f85-830e-c45a62d3c730 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:49.566028 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680f3181-0c71-4f85-830e-c45a62d3c730 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:49.567464 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680f3181-0c71-4f85-830e-c45a62d3c730 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:49.627876 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:49.671760 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 344/701] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:54:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 14:54:49.677275 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22db52-5b76-4c18-9410-2aec2707724c None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:49.708758 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f3181-0c71-4f85-830e-c45a62d3c730 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:49.819126 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22db52-5b76-4c18-9410-2aec2707724c None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:49.839629 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f3181-0c71-4f85-830e-c45a62d3c730 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:49.859485 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e010c3-5ffc-495a-8ab9-37112390cd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:49.860127 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e010c3-5ffc-495a-8ab9-37112390cd6a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:49.860400 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e010c3-5ffc-495a-8ab9-37112390cd6a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:49.860668 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e010c3-5ffc-495a-8ab9-37112390cd6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:49.875070 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e010c3-5ffc-495a-8ab9-37112390cd6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:49.876719 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e010c3-5ffc-495a-8ab9-37112390cd6a None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:49.882259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03846a-9902-4f6b-aa84-1f2d7758b212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:49.883042 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03846a-9902-4f6b-aa84-1f2d7758b212 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:49.883419 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03846a-9902-4f6b-aa84-1f2d7758b212 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:49.883795 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03846a-9902-4f6b-aa84-1f2d7758b212 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:49.895367 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-83e010c3-5ffc-495a-8ab9-37112390cd6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:49.895367 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:49.895367 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:49.895367 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:49.895367 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:49.895367 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:49.895367 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:49.895367 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:49.895367 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:49.895367 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:49.895367 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:49.895367 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:49.895367 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:49.895367 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:49.895367 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:49.895367 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:49.896964 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:49.896964 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:49.896964 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:49.896964 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:49.896964 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:49.896964 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:49.896964 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:49.896964 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:49.896964 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:49.896964 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:49.896964 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:49.896964 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:49.896964 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:49.896964 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:49.896964 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:49.896964 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:49.896964 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:49.898979 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:49.898979 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:49.898979 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:49.899317 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03846a-9902-4f6b-aa84-1f2d7758b212 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:49.899317 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03846a-9902-4f6b-aa84-1f2d7758b212 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:49.899973 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 358/702] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:54:49.902387 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e010c3-5ffc-495a-8ab9-37112390cd6a None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:49.938856 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-3b03846a-9902-4f6b-aa84-1f2d7758b212 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:49.938856 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:49.938856 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:49.938856 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:49.938856 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:49.938856 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:49.938856 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:49.938856 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:49.938856 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:49.938856 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:49.938856 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:49.938856 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:49.938856 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:49.938856 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:49.938856 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:49.938856 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:49.939891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:49.939891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:49.939891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:49.939891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:49.939891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:49.939891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:49.939891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:49.939891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:49.939891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:49.939891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:49.939891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:49.939891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:49.939891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:49.939891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:49.939891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:49.939891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:49.939891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:49.940787 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:49.940787 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:49.940787 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:49.950483 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 345/703] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:54:49.954251 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03846a-9902-4f6b-aa84-1f2d7758b212 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:49.959708 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:49.989277 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:50.091454 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e010c3-5ffc-495a-8ab9-37112390cd6a None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:50.124361 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03846a-9902-4f6b-aa84-1f2d7758b212 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:50.128782 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e0521-60e2-4657-9727-f4a3b49cc202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:50.129907 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e0521-60e2-4657-9727-f4a3b49cc202 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:50.130356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e0521-60e2-4657-9727-f4a3b49cc202 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:50.130795 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e0521-60e2-4657-9727-f4a3b49cc202 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:50.132038 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e0521-60e2-4657-9727-f4a3b49cc202 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:50.132512 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e0521-60e2-4657-9727-f4a3b49cc202 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:50.134467 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e0521-60e2-4657-9727-f4a3b49cc202 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:50.152437 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 359/704] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:54:50.153958 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42193ee8-400e-4ea1-b54d-f3e953b62150 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:50.154404 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42193ee8-400e-4ea1-b54d-f3e953b62150 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:50.154983 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42193ee8-400e-4ea1-b54d-f3e953b62150 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:50.160890 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 360/705] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:50.160890 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42193ee8-400e-4ea1-b54d-f3e953b62150 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:50.175898 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61574bf3-dd98-46b2-8cb3-36203666d685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:50.176519 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61574bf3-dd98-46b2-8cb3-36203666d685 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:50.176794 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61574bf3-dd98-46b2-8cb3-36203666d685 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:50.177068 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61574bf3-dd98-46b2-8cb3-36203666d685 None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:50.178217 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61574bf3-dd98-46b2-8cb3-36203666d685 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:50.178524 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61574bf3-dd98-46b2-8cb3-36203666d685 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:50.180280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61574bf3-dd98-46b2-8cb3-36203666d685 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:50.201035 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 346/706] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:54:50.201035 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa7b1f-a523-4e24-aee8-a5c30c8c5f0b None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:50.201035 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa7b1f-a523-4e24-aee8-a5c30c8c5f0b None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:50.201035 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa7b1f-a523-4e24-aee8-a5c30c8c5f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:50.224204 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-60fa7b1f-a523-4e24-aee8-a5c30c8c5f0b None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:50.228559 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-60fa7b1f-a523-4e24-aee8-a5c30c8c5f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:50.291183 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:50.305077 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42193ee8-400e-4ea1-b54d-f3e953b62150 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:50.318841 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 347/707] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:50.318841 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa7b1f-a523-4e24-aee8-a5c30c8c5f0b None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:50.318841 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:50.377254 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b9b21-5a6c-49d1-b444-0c0c17ee5f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:50.377812 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b9b21-5a6c-49d1-b444-0c0c17ee5f3c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:50.378086 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b9b21-5a6c-49d1-b444-0c0c17ee5f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:50.378354 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b9b21-5a6c-49d1-b444-0c0c17ee5f3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:50.388987 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6b9b21-5a6c-49d1-b444-0c0c17ee5f3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:50.391229 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6b9b21-5a6c-49d1-b444-0c0c17ee5f3c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:50.403486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-8c6b9b21-5a6c-49d1-b444-0c0c17ee5f3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:50.403486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:50.403486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:50.403486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:50.403486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:50.403486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:50.403486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:50.403486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:50.403486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:50.403486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:50.403486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:50.403486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:50.403486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:54:50.403486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:54:50.403486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:54:50.404445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:54:50.404445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:50.404445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:50.404445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:50.404445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:50.404445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:50.404445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:50.404445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:50.404445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:50.404445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:50.404445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:50.404445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:50.404445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:50.404445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:50.404445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:50.404445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:50.404445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:50.405930 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:54:50.405930 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:54:50.405930 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:54:50.405930 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:54:50.405930 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:54:50.405930 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:54:50.405930 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:50.405930 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:50.410301 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 361/708] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:54:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:50.411676 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b9b21-5a6c-49d1-b444-0c0c17ee5f3c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:50.520334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa7b1f-a523-4e24-aee8-a5c30c8c5f0b None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:50.526664 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b9b21-5a6c-49d1-b444-0c0c17ee5f3c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:50.566985 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb5f5ca-4b46-413f-938d-d2646b58d5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:50.567599 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5f5ca-4b46-413f-938d-d2646b58d5f6 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:50.567939 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5f5ca-4b46-413f-938d-d2646b58d5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:50.568336 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5f5ca-4b46-413f-938d-d2646b58d5f6 None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:50.569245 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb5f5ca-4b46-413f-938d-d2646b58d5f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:50.569573 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb5f5ca-4b46-413f-938d-d2646b58d5f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:50.571032 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb5f5ca-4b46-413f-938d-d2646b58d5f6 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:50.574525 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e426c-7e86-4d35-a3e3-4c9419fd5b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:50.575169 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e426c-7e86-4d35-a3e3-4c9419fd5b0e None admin] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:50.575450 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e426c-7e86-4d35-a3e3-4c9419fd5b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:50.575725 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e426c-7e86-4d35-a3e3-4c9419fd5b0e None admin] PATH_INFO: `/v3/users` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:50.576381 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645e426c-7e86-4d35-a3e3-4c9419fd5b0e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:50.578425 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645e426c-7e86-4d35-a3e3-4c9419fd5b0e None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:50.581903 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:50.591015 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 362/709] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:54:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:54:50.592890 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987cf05-0e46-4fe5-b607-cf9c84d2e48d None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:50.594000 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987cf05-0e46-4fe5-b607-cf9c84d2e48d None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:50.594536 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987cf05-0e46-4fe5-b607-cf9c84d2e48d None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:50.596039 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 363/710] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:50.596802 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987cf05-0e46-4fe5-b607-cf9c84d2e48d None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:50.608328 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 348/711] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 14:54:50 2022] POST /identity/v3/users => generated 254 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:54:50.609781 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57428db-8702-439e-b063-9f864fb2bdc7 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:50.610238 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57428db-8702-439e-b063-9f864fb2bdc7 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:50.610527 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57428db-8702-439e-b063-9f864fb2bdc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:50.636597 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-a57428db-8702-439e-b063-9f864fb2bdc7 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:50.638185 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-a57428db-8702-439e-b063-9f864fb2bdc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:50.706827 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:50.726990 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 349/712] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:50.772853 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987cf05-0e46-4fe5-b607-cf9c84d2e48d None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:50.821197 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57428db-8702-439e-b063-9f864fb2bdc7 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:50.827792 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94bd83-eb88-4a26-9115-8faa92a84482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:50.828400 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94bd83-eb88-4a26-9115-8faa92a84482 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:50.828675 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94bd83-eb88-4a26-9115-8faa92a84482 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:50.828943 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94bd83-eb88-4a26-9115-8faa92a84482 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:50.829665 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd94bd83-eb88-4a26-9115-8faa92a84482 None admin] RBAC: Adding query filter params (role.id=99e8fe7df8e7446680a81d4eaaa1cb68, scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176, user.id=fc0ea93084b0482fb2dd6d802987942f) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:50.829967 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd94bd83-eb88-4a26-9115-8faa92a84482 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:50.831439 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd94bd83-eb88-4a26-9115-8faa92a84482 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:50.841244 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 364/713] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:54:50 2022] GET /identity/v3/role_assignments?user.id=fc0ea93084b0482fb2dd6d802987942f&scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176&role.id=99e8fe7df8e7446680a81d4eaaa1cb68 => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:50.956370 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57428db-8702-439e-b063-9f864fb2bdc7 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:51.003844 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb4f009-e541-4434-94b3-a1111cfd1abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:51.004768 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4f009-e541-4434-94b3-a1111cfd1abb None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:51.005201 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4f009-e541-4434-94b3-a1111cfd1abb None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:51.005620 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4f009-e541-4434-94b3-a1111cfd1abb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:51.006402 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb4f009-e541-4434-94b3-a1111cfd1abb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:51.008061 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb4f009-e541-4434-94b3-a1111cfd1abb None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:51.022511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-2cb4f009-e541-4434-94b3-a1111cfd1abb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 14:54:51.022511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:51.022511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:54:51.022511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:54:51.022511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:54:51.022511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:54:51.022511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:54:51.022511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:54:51.022511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 14:54:51.022511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:51.022511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:54:51.022511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:51.022511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:51.022511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:51.022511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:51.022511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:51.022511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:51.023837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:51.023837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:51.023837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:51.023837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:51.023837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:51.023837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:51.023837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:54:51.023837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:54:51.023837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:54:51.023837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:54:51.023837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:51.023837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:51.023837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:51.023837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:51.023837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:51.023837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:51.024955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:51.024955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:51.024955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:51.024955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:51.024955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:51.024955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:51.024955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:51.024955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:51.024955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:51.024955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:51.024955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:54:51.024955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:54:51.024955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 14:54:51.024955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:51.027459 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 350/714] 10.222.0.33 () {56 vars in 1129 bytes} [Sun Sep 11 14:54:50 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:54:51.035139 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94bd83-eb88-4a26-9115-8faa92a84482 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:51.045039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb4f009-e541-4434-94b3-a1111cfd1abb None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:51.108818 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:51.129907 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:51.160509 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb4f009-e541-4434-94b3-a1111cfd1abb None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:51.170208 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94bd83-eb88-4a26-9115-8faa92a84482 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:51.213426 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777dad4-140c-4673-9e94-20e6dc9687c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:51.213979 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777dad4-140c-4673-9e94-20e6dc9687c3 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:51.214253 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777dad4-140c-4673-9e94-20e6dc9687c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:51.214523 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777dad4-140c-4673-9e94-20e6dc9687c3 None admin] PATH_INFO: `/v3/services/network` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:51.215128 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e777dad4-140c-4673-9e94-20e6dc9687c3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:51.216586 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e777dad4-140c-4673-9e94-20e6dc9687c3 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:51.230961 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-e777dad4-140c-4673-9e94-20e6dc9687c3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 14:54:51.230961 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:51.230961 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:54:51.230961 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:54:51.230961 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:54:51.230961 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:54:51.230961 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:54:51.230961 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:54:51.230961 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 14:54:51.230961 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:51.230961 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:54:51.230961 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:51.230961 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:51.230961 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:51.230961 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:51.230961 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:51.230961 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:51.232020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:51.232020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:51.232020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:51.232020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:51.232020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:51.232020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:51.232020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:54:51.232020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:54:51.232020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:54:51.232020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:54:51.232020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:51.232020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:51.232020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:51.232020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:51.232020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:51.232020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:51.232921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:51.232921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:51.232921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:51.232921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:51.232921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:51.232921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:51.232921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:51.232921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:51.232921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:51.232921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:51.232921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:54:51.232921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:54:51.232921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 14:54:51.232921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:51.233683 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 351/715] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:54:51 2022] GET /identity/v3/services/network => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:54:51.238738 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777dad4-140c-4673-9e94-20e6dc9687c3 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:51.250643 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117a504-73fd-49c5-a500-6f5824ed58d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:51.251292 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117a504-73fd-49c5-a500-6f5824ed58d4 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:51.251564 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117a504-73fd-49c5-a500-6f5824ed58d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:51.251833 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117a504-73fd-49c5-a500-6f5824ed58d4 None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:51.252466 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117a504-73fd-49c5-a500-6f5824ed58d4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:51.252759 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117a504-73fd-49c5-a500-6f5824ed58d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:51.254144 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117a504-73fd-49c5-a500-6f5824ed58d4 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:51.262343 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 365/716] 10.222.0.33 () {56 vars in 1124 bytes} [Sun Sep 11 14:54:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:51.269590 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117a504-73fd-49c5-a500-6f5824ed58d4 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:51.396788 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117a504-73fd-49c5-a500-6f5824ed58d4 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:51.434230 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:51.457784 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777dad4-140c-4673-9e94-20e6dc9687c3 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:51.465597 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42104d8d-20a3-47d0-b35b-1d87d0b82886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:51.466152 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42104d8d-20a3-47d0-b35b-1d87d0b82886 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:51.466433 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42104d8d-20a3-47d0-b35b-1d87d0b82886 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:51.466709 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42104d8d-20a3-47d0-b35b-1d87d0b82886 None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:51.467608 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42104d8d-20a3-47d0-b35b-1d87d0b82886 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:51.467918 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42104d8d-20a3-47d0-b35b-1d87d0b82886 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:51.469427 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42104d8d-20a3-47d0-b35b-1d87d0b82886 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:51.479223 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:51.491827 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 366/717] 10.222.0.33 () {56 vars in 1124 bytes} [Sun Sep 11 14:54:51 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:54:51.510292 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ec6d3-41a6-4b94-8f55-f3a06d0db348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:51.511354 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ec6d3-41a6-4b94-8f55-f3a06d0db348 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:51.511845 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ec6d3-41a6-4b94-8f55-f3a06d0db348 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:51.512377 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ec6d3-41a6-4b94-8f55-f3a06d0db348 None admin] PATH_INFO: `/v3/services` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:51.513241 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459ec6d3-41a6-4b94-8f55-f3a06d0db348 None admin] RBAC: Adding query filter params (name=network) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:51.513770 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459ec6d3-41a6-4b94-8f55-f3a06d0db348 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:51.515980 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459ec6d3-41a6-4b94-8f55-f3a06d0db348 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:51.526493 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 352/718] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 11 14:54:51 2022] GET /identity/v3/services?name=network => generated 124 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:51.531884 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42104d8d-20a3-47d0-b35b-1d87d0b82886 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:51.687400 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42104d8d-20a3-47d0-b35b-1d87d0b82886 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:51.724812 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2922db8d-5d83-494e-8539-3c83b52f2b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:51.725798 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922db8d-5d83-494e-8539-3c83b52f2b7d None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:51.726286 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922db8d-5d83-494e-8539-3c83b52f2b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:51.726780 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922db8d-5d83-494e-8539-3c83b52f2b7d None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:51.727690 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2922db8d-5d83-494e-8539-3c83b52f2b7d None admin] RBAC: Adding query filter params (type=network) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:51.728196 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2922db8d-5d83-494e-8539-3c83b52f2b7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:51.729829 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2922db8d-5d83-494e-8539-3c83b52f2b7d None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:51.736314 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 367/719] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 11 14:54:51 2022] GET /identity/v3/services?type=network => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:52.190685 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c817b2-b7d0-43eb-b778-88dc0645fecb None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:52.191727 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c817b2-b7d0-43eb-b778-88dc0645fecb None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:52.192222 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c817b2-b7d0-43eb-b778-88dc0645fecb None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:52.194063 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 353/720] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:52.199405 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb51c80-ae0d-4703-82d2-a49b054819cb None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:52.199824 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb51c80-ae0d-4703-82d2-a49b054819cb None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:52.199952 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb51c80-ae0d-4703-82d2-a49b054819cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:52.221319 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-fcb51c80-ae0d-4703-82d2-a49b054819cb None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:52.222847 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-fcb51c80-ae0d-4703-82d2-a49b054819cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:52.276334 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:52.299629 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 368/721] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:52.572246 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c817b2-b7d0-43eb-b778-88dc0645fecb None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:52.703266 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c817b2-b7d0-43eb-b778-88dc0645fecb None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:52.757496 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1756a704-4d0b-4fb0-a214-0dcde76e157e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:52.758518 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756a704-4d0b-4fb0-a214-0dcde76e157e None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:52.759074 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756a704-4d0b-4fb0-a214-0dcde76e157e None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:52.759545 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756a704-4d0b-4fb0-a214-0dcde76e157e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:52.773084 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1756a704-4d0b-4fb0-a214-0dcde76e157e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:52.778850 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-1756a704-4d0b-4fb0-a214-0dcde76e157e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:54:52.778850 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:52.778850 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:52.778850 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:52.778850 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:52.778850 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:52.778850 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:52.778850 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:52.778850 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:52.778850 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:52.778850 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:52.778850 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:52.778850 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:52.778850 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:52.778850 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:52.778850 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:52.780457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:52.780457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:52.780457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:52.780457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:52.780457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:52.780457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:52.780457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:52.780457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:52.780457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:52.780457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:52.780457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:52.780457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:52.780457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:52.780457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:52.780457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:52.780457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:52.781384 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:52.781384 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:52.781384 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:52.781384 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:52.781384 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:52.781384 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:52.781384 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:52.781384 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:52.781384 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:52.781384 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:52.781384 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:54:52.781384 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:52.783702 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 354/722] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:54:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:52.789623 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb51c80-ae0d-4703-82d2-a49b054819cb None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:52.935040 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb51c80-ae0d-4703-82d2-a49b054819cb None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:52.980961 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d46e82c-e29a-4071-b28d-03b134d36ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:52.981955 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46e82c-e29a-4071-b28d-03b134d36ab5 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:52.982450 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46e82c-e29a-4071-b28d-03b134d36ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:52.983470 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46e82c-e29a-4071-b28d-03b134d36ab5 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:52.984335 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d46e82c-e29a-4071-b28d-03b134d36ab5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:52.984831 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d46e82c-e29a-4071-b28d-03b134d36ab5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:52.986419 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d46e82c-e29a-4071-b28d-03b134d36ab5 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:53.001100 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 369/723] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:54:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:53.006673 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1756a704-4d0b-4fb0-a214-0dcde76e157e None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:53.064033 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:53.141239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1756a704-4d0b-4fb0-a214-0dcde76e157e None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:53.191677 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4266979-9cd4-4468-8323-d4a0f00d2cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:53.192317 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4266979-9cd4-4468-8323-d4a0f00d2cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:53.192549 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4266979-9cd4-4468-8323-d4a0f00d2cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:53.192764 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4266979-9cd4-4468-8323-d4a0f00d2cb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:53.214533 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4266979-9cd4-4468-8323-d4a0f00d2cb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:53.216027 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4266979-9cd4-4468-8323-d4a0f00d2cb1 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:53.235457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-c4266979-9cd4-4468-8323-d4a0f00d2cb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:53.235457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:53.235457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:53.235457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:53.235457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:53.235457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:53.235457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:53.235457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:53.235457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:53.235457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:53.235457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:53.235457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:53.235457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:53.235457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:53.235457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:53.235457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:53.236488 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:53.236488 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:53.236488 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:53.236488 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:53.236488 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:53.236488 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:53.236488 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:53.236488 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:53.236488 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:53.236488 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:53.236488 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:53.236488 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:53.236488 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:53.236488 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:53.236488 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:53.236488 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:53.236488 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:53.237480 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:53.237480 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:53.237480 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:53.237480 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 355/724] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:53.243838 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d46e82c-e29a-4071-b28d-03b134d36ab5 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:53.331972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:53.402895 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d46e82c-e29a-4071-b28d-03b134d36ab5 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:53.447175 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c5bc6-9038-4d04-9609-282881c4fdba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:53.448177 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c5bc6-9038-4d04-9609-282881c4fdba None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:53.448719 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c5bc6-9038-4d04-9609-282881c4fdba None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:53.449198 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c5bc6-9038-4d04-9609-282881c4fdba None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:53.450085 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c5bc6-9038-4d04-9609-282881c4fdba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:53.450612 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c5bc6-9038-4d04-9609-282881c4fdba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:53.452389 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c5bc6-9038-4d04-9609-282881c4fdba None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:53.475009 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 370/725] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:53.477998 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4266979-9cd4-4468-8323-d4a0f00d2cb1 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:53.576802 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c07f91-cfd3-4453-bc19-9cca2d3e2c01 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:53.577325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c07f91-cfd3-4453-bc19-9cca2d3e2c01 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:53.577708 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c07f91-cfd3-4453-bc19-9cca2d3e2c01 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:53.580937 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 371/726] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:54:53.585636 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1671543-8d7d-47ee-83f8-96c11f770500 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:53.585916 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1671543-8d7d-47ee-83f8-96c11f770500 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:53.586189 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1671543-8d7d-47ee-83f8-96c11f770500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:53.607663 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-e1671543-8d7d-47ee-83f8-96c11f770500 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:53.609221 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-e1671543-8d7d-47ee-83f8-96c11f770500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:53.646008 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:53.668484 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4266979-9cd4-4468-8323-d4a0f00d2cb1 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:53.694908 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 372/727] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:53.696329 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d408ee-6c56-4f6d-968d-8d8dfcaa1c54 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:53.696614 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d408ee-6c56-4f6d-968d-8d8dfcaa1c54 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:53.697677 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d408ee-6c56-4f6d-968d-8d8dfcaa1c54 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:53.700744 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 373/728] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:53.709855 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c46e46-4f81-47b7-9664-70189be2ab35 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:53.710142 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c46e46-4f81-47b7-9664-70189be2ab35 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:53.710421 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c46e46-4f81-47b7-9664-70189be2ab35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:53.737369 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-40c46e46-4f81-47b7-9664-70189be2ab35 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:53.738620 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-40c46e46-4f81-47b7-9664-70189be2ab35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:53.757715 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb3b55b-068f-48fd-8db0-d50573bc306a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:53.759451 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3b55b-068f-48fd-8db0-d50573bc306a None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:53.759738 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3b55b-068f-48fd-8db0-d50573bc306a None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:53.760013 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3b55b-068f-48fd-8db0-d50573bc306a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:53.775276 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb3b55b-068f-48fd-8db0-d50573bc306a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:53.776736 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb3b55b-068f-48fd-8db0-d50573bc306a None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:53.783646 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:53.805589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-fcb3b55b-068f-48fd-8db0-d50573bc306a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 14:54:53.805589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:53.805589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:53.805589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:53.805589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:53.805589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:53.805589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:53.805589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:53.805589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:53.805589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:53.805589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:53.805589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:53.805589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:54:53.805589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:54:53.805589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:54:53.805589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:54:53.806589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:53.806589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:53.806589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:54:53.806589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:53.806589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:54:53.806589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:53.806589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:53.806589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:53.806589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:53.806589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:53.806589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:53.806589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:53.806589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:53.806589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:53.806589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:53.806589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:53.806589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:53.806589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:53.807565 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:53.807565 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:53.807565 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:54:53.807565 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:54:53.807565 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:54:53.807565 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:54:53.807565 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:54:53.807565 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:54:53.807565 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 14:54:53.807565 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:53.809890 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 356/729] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:54:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:53.814707 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb3b55b-068f-48fd-8db0-d50573bc306a None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:53.856326 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 374/730] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:54.013535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c46e46-4f81-47b7-9664-70189be2ab35 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:54.041608 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb3b55b-068f-48fd-8db0-d50573bc306a None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:54.115619 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52897c3-d9a6-4771-a0b2-108199cb29c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:54.116268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52897c3-d9a6-4771-a0b2-108199cb29c1 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:54.116550 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52897c3-d9a6-4771-a0b2-108199cb29c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:54.116824 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52897c3-d9a6-4771-a0b2-108199cb29c1 None admin] PATH_INFO: `/v3/users` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:54.117569 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52897c3-d9a6-4771-a0b2-108199cb29c1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:54.117880 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52897c3-d9a6-4771-a0b2-108199cb29c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:54.119372 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52897c3-d9a6-4771-a0b2-108199cb29c1 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:54.155271 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 357/731] 10.222.0.33 () {56 vars in 1135 bytes} [Sun Sep 11 14:54:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:54.156144 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52897c3-d9a6-4771-a0b2-108199cb29c1 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:54.165777 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c46e46-4f81-47b7-9664-70189be2ab35 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:54.169962 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:54.213705 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53073b68-e605-4477-aae5-56fbc2d90e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:54.214369 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53073b68-e605-4477-aae5-56fbc2d90e28 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:54.214718 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53073b68-e605-4477-aae5-56fbc2d90e28 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:54.215183 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53073b68-e605-4477-aae5-56fbc2d90e28 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:54.227272 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53073b68-e605-4477-aae5-56fbc2d90e28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:54.232549 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-53073b68-e605-4477-aae5-56fbc2d90e28 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:54:54.232549 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:54.232549 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:54.232549 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:54.232549 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:54.232549 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:54.232549 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:54.232549 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:54.232549 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:54.232549 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:54.232549 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:54.232549 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:54.232549 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:54.232549 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:54.232549 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:54.232549 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:54.234058 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:54.234058 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:54.234058 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:54.234058 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:54.234058 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:54.234058 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:54.234058 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:54.234058 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:54.234058 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:54.234058 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:54.234058 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:54.234058 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:54.234058 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:54.234058 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:54.234058 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:54.234058 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:54.239570 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:54.239570 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:54.239570 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:54.239570 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:54.239570 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:54.239570 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:54.239570 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:54.239570 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:54.239570 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:54.239570 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:54.239570 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:54:54.239570 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:54.239570 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 375/732] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:54:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:54:54.239570 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53073b68-e605-4477-aae5-56fbc2d90e28 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:54.267471 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:54.289822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52897c3-d9a6-4771-a0b2-108199cb29c1 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:54.329564 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb27620-09c8-4dc2-bd34-51e122f0cc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:54.330200 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb27620-09c8-4dc2-bd34-51e122f0cc80 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:54.330472 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb27620-09c8-4dc2-bd34-51e122f0cc80 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:54.330749 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb27620-09c8-4dc2-bd34-51e122f0cc80 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:54.342006 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb27620-09c8-4dc2-bd34-51e122f0cc80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:54.345755 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53073b68-e605-4477-aae5-56fbc2d90e28 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:54.347487 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-beb27620-09c8-4dc2-bd34-51e122f0cc80 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:54:54.347487 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:54.347487 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:54.347487 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:54.347487 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:54.347487 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:54.347487 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:54.347487 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:54.347487 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:54.347487 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:54.347487 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:54.347487 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:54.347487 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:54.347487 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:54.347487 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:54.347487 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:54.348416 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:54.348416 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:54.348416 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:54.348416 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:54.348416 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:54.348416 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:54.348416 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:54.348416 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:54.348416 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:54.348416 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:54.348416 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:54.348416 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:54.348416 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:54.348416 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:54.348416 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:54.348416 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:54.354524 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:54.354524 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:54.354524 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:54.354524 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:54.354524 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:54.354524 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:54.354524 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:54.354524 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:54.354524 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:54.354524 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:54.354524 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:54:54.354524 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:54.354524 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 358/733] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 14:54:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:54.354524 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb27620-09c8-4dc2-bd34-51e122f0cc80 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:54.383562 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f810d255-729c-4e04-a329-380d6a437a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:54.384170 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810d255-729c-4e04-a329-380d6a437a9b None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:54.384451 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810d255-729c-4e04-a329-380d6a437a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:54.384723 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810d255-729c-4e04-a329-380d6a437a9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:54.427014 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:54.458457 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f810d255-729c-4e04-a329-380d6a437a9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:54.460014 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f810d255-729c-4e04-a329-380d6a437a9b None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:54.463033 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb27620-09c8-4dc2-bd34-51e122f0cc80 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:54.491454 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:54.504647 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a000b-5387-4be1-ad23-d4fd4443c53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:54.505260 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a000b-5387-4be1-ad23-d4fd4443c53a None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:54.505535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a000b-5387-4be1-ad23-d4fd4443c53a None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:54.505882 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a000b-5387-4be1-ad23-d4fd4443c53a None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:54.506543 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a000b-5387-4be1-ad23-d4fd4443c53a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:54.506870 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a000b-5387-4be1-ad23-d4fd4443c53a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:54.508359 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a000b-5387-4be1-ad23-d4fd4443c53a None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:54.517387 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 359/734] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:54:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:54.518697 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5bd8c-c626-4d3e-89fc-8e92eee2946b None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:54.519015 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5bd8c-c626-4d3e-89fc-8e92eee2946b None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:54.519291 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5bd8c-c626-4d3e-89fc-8e92eee2946b None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:54.522838 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 360/735] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:54.522838 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5bd8c-c626-4d3e-89fc-8e92eee2946b None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:54.584935 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 376/736] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:54:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:54:54.585792 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f810d255-729c-4e04-a329-380d6a437a9b None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:54.626967 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5bd8c-c626-4d3e-89fc-8e92eee2946b None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:54.656840 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:54.662243 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d5865-ac19-46cf-a706-7ca693649c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:54.662872 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d5865-ac19-46cf-a706-7ca693649c79 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:54.663154 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d5865-ac19-46cf-a706-7ca693649c79 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:54.663430 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d5865-ac19-46cf-a706-7ca693649c79 None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:54.664082 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4d5865-ac19-46cf-a706-7ca693649c79 None admin] RBAC: Adding query filter params (name=service) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:54.664385 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4d5865-ac19-46cf-a706-7ca693649c79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:54.665760 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4d5865-ac19-46cf-a706-7ca693649c79 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:54.674925 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 361/737] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 14:54:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:54.675940 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f1923-109f-4c62-8cbc-aa5c82a0e931 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:54.676220 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f1923-109f-4c62-8cbc-aa5c82a0e931 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:54.676491 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f1923-109f-4c62-8cbc-aa5c82a0e931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:54.688883 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f810d255-729c-4e04-a329-380d6a437a9b None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:54.694129 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-520f1923-109f-4c62-8cbc-aa5c82a0e931 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:54.695355 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-520f1923-109f-4c62-8cbc-aa5c82a0e931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:54.715480 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:54.723827 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d874d59-a8fe-4ed1-92fe-42cd07335c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:54.724452 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d874d59-a8fe-4ed1-92fe-42cd07335c43 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:54.724726 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d874d59-a8fe-4ed1-92fe-42cd07335c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:54.725000 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d874d59-a8fe-4ed1-92fe-42cd07335c43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:54.735304 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d874d59-a8fe-4ed1-92fe-42cd07335c43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:54.736762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d874d59-a8fe-4ed1-92fe-42cd07335c43 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:54.752071 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-7d874d59-a8fe-4ed1-92fe-42cd07335c43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:54.752071 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:54.752071 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:54.752071 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:54.752071 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:54.752071 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:54.752071 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:54.752071 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:54.752071 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:54.752071 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:54.752071 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:54.752071 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:54.752071 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:54.752071 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:54.752071 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:54.752071 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:54.753028 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:54.753028 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:54.753028 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:54.753028 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:54.753028 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:54.753028 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:54.753028 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:54.753028 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:54.753028 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:54.753028 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:54.753028 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:54.753028 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:54.753028 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:54.753028 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:54.753028 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:54.753028 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:54.753028 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:54.754076 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:54.754076 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:54.754076 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:54.754076 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 377/738] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:54.755060 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d42cc-d614-45c5-ad49-c17db53b05b0 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:54.755060 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d42cc-d614-45c5-ad49-c17db53b05b0 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:54.755060 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d42cc-d614-45c5-ad49-c17db53b05b0 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:54.756189 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 378/739] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:54.756714 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d42cc-d614-45c5-ad49-c17db53b05b0 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:54.766752 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 362/740] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:54.767616 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f1923-109f-4c62-8cbc-aa5c82a0e931 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:54.863638 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d42cc-d614-45c5-ad49-c17db53b05b0 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:54.875682 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f1923-109f-4c62-8cbc-aa5c82a0e931 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:54.898894 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95768d52-6f7a-44fe-b5f4-97bea04a7045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:54.899527 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95768d52-6f7a-44fe-b5f4-97bea04a7045 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:54.899817 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95768d52-6f7a-44fe-b5f4-97bea04a7045 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:54.900082 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95768d52-6f7a-44fe-b5f4-97bea04a7045 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:54.901237 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:54.913601 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95768d52-6f7a-44fe-b5f4-97bea04a7045 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:54.914700 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3fb741-7fe0-460e-86e2-27435634c617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:54.915123 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95768d52-6f7a-44fe-b5f4-97bea04a7045 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:54.915369 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3fb741-7fe0-460e-86e2-27435634c617 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:54.915674 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3fb741-7fe0-460e-86e2-27435634c617 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:54.915984 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3fb741-7fe0-460e-86e2-27435634c617 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:54.927449 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3fb741-7fe0-460e-86e2-27435634c617 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:54.928891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3fb741-7fe0-460e-86e2-27435634c617 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:54.930501 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-95768d52-6f7a-44fe-b5f4-97bea04a7045 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:54.930501 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:54.930501 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:54.930501 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:54.930501 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:54.930501 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:54.930501 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:54.930501 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:54.930501 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:54.930501 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:54.930501 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:54.930501 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:54.930501 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:54.930501 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:54.930501 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:54.930501 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:54.931667 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:54.931667 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:54.931667 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:54.931667 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:54.931667 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:54.931667 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:54.931667 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:54.931667 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:54.931667 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:54.931667 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:54.931667 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:54.931667 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:54.931667 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:54.931667 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:54.931667 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:54.931667 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:54.931667 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:54.932568 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:54.932568 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:54.932568 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:54.932568 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 379/741] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:54.932568 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95768d52-6f7a-44fe-b5f4-97bea04a7045 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:54.951301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-dd3fb741-7fe0-460e-86e2-27435634c617 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:54.951301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:54.951301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:54.951301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:54.951301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:54.951301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:54.951301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:54.951301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:54.951301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:54.951301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:54.951301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:54.951301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:54.951301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:54.951301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:54.951301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:54.951301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:54.952586 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:54.952586 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:54.952586 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:54.952586 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:54.952586 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:54.952586 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:54.952586 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:54.952586 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:54.952586 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:54.952586 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:54.952586 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:54.952586 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:54.952586 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:54.952586 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:54.952586 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:54.952586 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:54.952586 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:54.953935 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:54.953935 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:54.953935 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:54.953935 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 363/742] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:54.958957 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44923c62-e877-435d-8f27-03551555c304 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:54.959249 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44923c62-e877-435d-8f27-03551555c304 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:54.959528 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44923c62-e877-435d-8f27-03551555c304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:54.968882 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:54.980879 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-44923c62-e877-435d-8f27-03551555c304 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:54.982508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-44923c62-e877-435d-8f27-03551555c304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:55.047254 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95768d52-6f7a-44fe-b5f4-97bea04a7045 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:55.053689 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 364/743] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:54:55.054595 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44923c62-e877-435d-8f27-03551555c304 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:55.085369 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b41830-7a90-4bc7-bf4c-7c2f36431a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:55.086208 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b41830-7a90-4bc7-bf4c-7c2f36431a28 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:55.086647 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b41830-7a90-4bc7-bf4c-7c2f36431a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:55.086956 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b41830-7a90-4bc7-bf4c-7c2f36431a28 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:55.087619 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b41830-7a90-4bc7-bf4c-7c2f36431a28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:55.087920 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b41830-7a90-4bc7-bf4c-7c2f36431a28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:55.089295 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b41830-7a90-4bc7-bf4c-7c2f36431a28 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:55.105759 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 380/744] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:55.106484 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b41830-7a90-4bc7-bf4c-7c2f36431a28 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:55.159343 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44923c62-e877-435d-8f27-03551555c304 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:55.168350 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:55.200772 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:55.200944 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da76ccb-155e-4705-9adf-ee89e77ec83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:55.201372 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da76ccb-155e-4705-9adf-ee89e77ec83d None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:55.201724 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da76ccb-155e-4705-9adf-ee89e77ec83d None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:55.202091 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da76ccb-155e-4705-9adf-ee89e77ec83d None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:55.202926 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da76ccb-155e-4705-9adf-ee89e77ec83d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:55.203263 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da76ccb-155e-4705-9adf-ee89e77ec83d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:55.204711 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da76ccb-155e-4705-9adf-ee89e77ec83d None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:55.210892 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b41830-7a90-4bc7-bf4c-7c2f36431a28 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:55.218254 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 365/745] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:55.219593 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da76ccb-155e-4705-9adf-ee89e77ec83d None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:55.247141 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f75dfe-fd42-4a01-965b-7887de4828fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:55.248029 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f75dfe-fd42-4a01-965b-7887de4828fc None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:55.249022 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f75dfe-fd42-4a01-965b-7887de4828fc None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:55.249353 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f75dfe-fd42-4a01-965b-7887de4828fc None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:55.250072 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f75dfe-fd42-4a01-965b-7887de4828fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:55.250370 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f75dfe-fd42-4a01-965b-7887de4828fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:55.251775 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f75dfe-fd42-4a01-965b-7887de4828fc None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:55.269240 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 381/746] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:55.269240 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f75dfe-fd42-4a01-965b-7887de4828fc None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:55.324817 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da76ccb-155e-4705-9adf-ee89e77ec83d None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:55.361185 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a78dd-67a6-49e8-915a-006ad2de563b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:55.361791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a78dd-67a6-49e8-915a-006ad2de563b None admin] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:55.362074 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a78dd-67a6-49e8-915a-006ad2de563b None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:55.362346 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a78dd-67a6-49e8-915a-006ad2de563b None admin] PATH_INFO: `/v3/services` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:55.363027 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5a78dd-67a6-49e8-915a-006ad2de563b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:55.364411 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5a78dd-67a6-49e8-915a-006ad2de563b None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:55.375276 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f75dfe-fd42-4a01-965b-7887de4828fc None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:55.381732 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 366/747] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 11 14:54:55 2022] POST /identity/v3/services => generated 251 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:54:55.382469 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a78dd-67a6-49e8-915a-006ad2de563b None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:55.414923 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:55.419532 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1520d96-eb0f-409b-b513-531957381b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:55.421194 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1520d96-eb0f-409b-b513-531957381b96 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:55.421489 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1520d96-eb0f-409b-b513-531957381b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:55.421767 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1520d96-eb0f-409b-b513-531957381b96 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:55.432007 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1520d96-eb0f-409b-b513-531957381b96 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:55.433853 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1520d96-eb0f-409b-b513-531957381b96 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:55.448860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-d1520d96-eb0f-409b-b513-531957381b96 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 14:54:55.448860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:55.448860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:55.448860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:55.448860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:55.448860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:55.448860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:55.448860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:55.448860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:55.448860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:55.448860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:55.448860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:55.448860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:54:55.448860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:54:55.448860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:54:55.448860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:54:55.450101 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:55.450101 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:55.450101 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:54:55.450101 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:55.450101 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:54:55.450101 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:55.450101 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:55.450101 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:55.450101 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:55.450101 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:55.450101 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:55.450101 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:55.450101 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:55.450101 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:55.450101 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:55.450101 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:55.450101 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:55.450101 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:55.451826 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:55.451826 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:55.451826 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:54:55.451826 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:54:55.451826 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:54:55.451826 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:54:55.451826 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:54:55.451826 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:54:55.451826 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 14:54:55.451826 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:55.451826 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 382/748] 10.222.0.33 () {56 vars in 1093 bytes} [Sun Sep 11 14:54:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:55.451826 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1520d96-eb0f-409b-b513-531957381b96 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:55.465468 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:55.509370 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a78dd-67a6-49e8-915a-006ad2de563b None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:55.550770 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec5801-198a-434b-b994-446fb66c51f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:55.551593 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec5801-198a-434b-b994-446fb66c51f5 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:55.551938 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec5801-198a-434b-b994-446fb66c51f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:55.552589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec5801-198a-434b-b994-446fb66c51f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:55.561873 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec5801-198a-434b-b994-446fb66c51f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:55.563378 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec5801-198a-434b-b994-446fb66c51f5 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:55.575663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-0eec5801-198a-434b-b994-446fb66c51f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:55.575663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:55.575663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:55.575663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:55.575663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:55.575663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:55.575663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:55.575663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:55.575663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:55.575663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:55.575663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:55.575663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:55.575663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:54:55.575663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:54:55.575663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:54:55.576643 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:54:55.576643 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:55.576643 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:55.576643 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:55.576643 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:55.576643 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:55.576643 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:55.576643 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:55.576643 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:55.576643 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:55.576643 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:55.576643 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:55.576643 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:55.576643 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:55.576643 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:55.576643 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:55.576643 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:55.578747 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:54:55.578747 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:54:55.578747 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:54:55.578747 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:54:55.578747 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:54:55.578747 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:54:55.578747 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:55.578747 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:55.578747 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 367/749] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:54:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:55.578747 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec5801-198a-434b-b994-446fb66c51f5 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:55.587519 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1520d96-eb0f-409b-b513-531957381b96 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:55.626938 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279462c8-8226-4112-9f8d-37be6c9fe345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:55.627571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279462c8-8226-4112-9f8d-37be6c9fe345 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:55.627851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279462c8-8226-4112-9f8d-37be6c9fe345 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:55.628125 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279462c8-8226-4112-9f8d-37be6c9fe345 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:55.636888 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279462c8-8226-4112-9f8d-37be6c9fe345 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:55.638641 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279462c8-8226-4112-9f8d-37be6c9fe345 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:55.652261 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-279462c8-8226-4112-9f8d-37be6c9fe345 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 14:54:55.652261 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:55.652261 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:55.652261 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:55.652261 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:55.652261 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:55.652261 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:55.652261 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:55.652261 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:55.652261 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:55.652261 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:55.652261 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:55.652261 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:54:55.652261 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:54:55.652261 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:54:55.652261 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:54:55.658657 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:55.658657 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:55.658657 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:54:55.658657 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:55.658657 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:54:55.658657 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:54:55.658657 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:55.658657 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:55.658657 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:55.658657 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:55.658657 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:55.658657 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:55.658657 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:55.658657 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:55.658657 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:55.658657 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:55.658657 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:55.658657 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:55.659887 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:55.659887 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:55.659887 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:54:55.659887 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:54:55.659887 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:54:55.659887 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:54:55.659887 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:54:55.659887 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:54:55.659887 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 14:54:55.659887 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:55.659887 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 383/750] 10.222.0.33 () {56 vars in 1117 bytes} [Sun Sep 11 14:54:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:55.659887 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279462c8-8226-4112-9f8d-37be6c9fe345 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:55.680210 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:55.697725 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec5801-198a-434b-b994-446fb66c51f5 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:55.725932 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:55.735991 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87685fe2-5771-46f0-833e-efbca2db7222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:55.736626 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87685fe2-5771-46f0-833e-efbca2db7222 None admin] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:55.736904 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87685fe2-5771-46f0-833e-efbca2db7222 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:55.737183 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87685fe2-5771-46f0-833e-efbca2db7222 None admin] PATH_INFO: `/v3/services` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:55.737849 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87685fe2-5771-46f0-833e-efbca2db7222 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:55.739267 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87685fe2-5771-46f0-833e-efbca2db7222 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:55.756014 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 368/751] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 11 14:54:55 2022] POST /identity/v3/services => generated 240 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:54:55.756837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87685fe2-5771-46f0-833e-efbca2db7222 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:55.783330 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279462c8-8226-4112-9f8d-37be6c9fe345 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:55.841565 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64c3c3d-5f19-4b8c-8bd9-977ef53d0f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:55.842281 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c3c3d-5f19-4b8c-8bd9-977ef53d0f37 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:55.842772 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c3c3d-5f19-4b8c-8bd9-977ef53d0f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:55.842899 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c3c3d-5f19-4b8c-8bd9-977ef53d0f37 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:55.843725 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64c3c3d-5f19-4b8c-8bd9-977ef53d0f37 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:55.844091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64c3c3d-5f19-4b8c-8bd9-977ef53d0f37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:55.846161 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64c3c3d-5f19-4b8c-8bd9-977ef53d0f37 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:55.866969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 384/752] 10.222.0.33 () {56 vars in 1133 bytes} [Sun Sep 11 14:54:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:55.867931 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64c3c3d-5f19-4b8c-8bd9-977ef53d0f37 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:55.895832 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87685fe2-5771-46f0-833e-efbca2db7222 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:55.943214 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e832966-0a2d-457a-ade5-71f0a67db83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:55.944626 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e832966-0a2d-457a-ade5-71f0a67db83d None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:55.944626 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e832966-0a2d-457a-ade5-71f0a67db83d None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:55.944626 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e832966-0a2d-457a-ade5-71f0a67db83d None admin] PATH_INFO: `/v3/projects` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:55.945268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e832966-0a2d-457a-ade5-71f0a67db83d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:55.945268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e832966-0a2d-457a-ade5-71f0a67db83d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:55.948170 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e832966-0a2d-457a-ade5-71f0a67db83d None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:55.960599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 369/753] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:54:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:55.961450 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e832966-0a2d-457a-ade5-71f0a67db83d None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:55.976600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:56.026736 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64c3c3d-5f19-4b8c-8bd9-977ef53d0f37 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:56.052542 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:56.067038 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f188d-70a4-407b-add1-562eeb9af5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:56.067797 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f188d-70a4-407b-add1-562eeb9af5d7 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:56.068182 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f188d-70a4-407b-add1-562eeb9af5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:56.068561 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f188d-70a4-407b-add1-562eeb9af5d7 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:56.069474 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f188d-70a4-407b-add1-562eeb9af5d7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:56.069884 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f188d-70a4-407b-add1-562eeb9af5d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:56.071460 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f188d-70a4-407b-add1-562eeb9af5d7 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:56.097861 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 385/754] 10.222.0.33 () {56 vars in 1149 bytes} [Sun Sep 11 14:54:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:56.098972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f188d-70a4-407b-add1-562eeb9af5d7 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:56.104671 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e832966-0a2d-457a-ade5-71f0a67db83d None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:56.144580 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5fbe6b-9a10-448a-96e4-67badcc9e6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:56.145390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5fbe6b-9a10-448a-96e4-67badcc9e6b7 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:56.146473 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5fbe6b-9a10-448a-96e4-67badcc9e6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:56.146473 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5fbe6b-9a10-448a-96e4-67badcc9e6b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:56.157730 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5fbe6b-9a10-448a-96e4-67badcc9e6b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:56.159463 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5fbe6b-9a10-448a-96e4-67badcc9e6b7 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:56.176104 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-1b5fbe6b-9a10-448a-96e4-67badcc9e6b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:56.176104 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:56.176104 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:56.176104 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:56.176104 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:56.176104 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:56.176104 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:56.176104 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:56.176104 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:56.176104 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:56.176104 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:56.176104 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:56.176104 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:56.176104 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:56.176104 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:56.176104 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:56.177276 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:56.177276 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:56.177276 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:56.177276 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:56.177276 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:56.177276 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:56.177276 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:56.177276 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:56.177276 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:56.177276 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:56.177276 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:56.177276 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:56.177276 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:56.177276 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:56.177276 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:56.177276 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:56.177276 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:56.178394 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:56.178394 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:56.178394 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:56.179814 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 370/755] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:56.180822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5fbe6b-9a10-448a-96e4-67badcc9e6b7 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:56.227617 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f188d-70a4-407b-add1-562eeb9af5d7 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:56.259048 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:56.266797 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32fac9f-cc7b-4ead-b7ee-2a775c56679b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:56.267441 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fac9f-cc7b-4ead-b7ee-2a775c56679b None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:56.267718 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fac9f-cc7b-4ead-b7ee-2a775c56679b None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:56.267987 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fac9f-cc7b-4ead-b7ee-2a775c56679b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:56.268714 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32fac9f-cc7b-4ead-b7ee-2a775c56679b None admin] RBAC: Adding query filter params (role.id=99e8fe7df8e7446680a81d4eaaa1cb68, scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176, user.id=698f1293294d45b89e04d85649f454d0) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:56.269020 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32fac9f-cc7b-4ead-b7ee-2a775c56679b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:56.270530 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32fac9f-cc7b-4ead-b7ee-2a775c56679b None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:56.286446 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 386/756] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:54:56 2022] GET /identity/v3/role_assignments?user.id=698f1293294d45b89e04d85649f454d0&scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176&role.id=99e8fe7df8e7446680a81d4eaaa1cb68 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:56.286446 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32fac9f-cc7b-4ead-b7ee-2a775c56679b None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:56.344911 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:56.379190 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5fbe6b-9a10-448a-96e4-67badcc9e6b7 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:56.412860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32fac9f-cc7b-4ead-b7ee-2a775c56679b None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:56.453272 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98d24b-cfd3-4fb9-b0f8-ade285c5f42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:56.453896 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98d24b-cfd3-4fb9-b0f8-ade285c5f42c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:56.454175 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98d24b-cfd3-4fb9-b0f8-ade285c5f42c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:56.454446 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98d24b-cfd3-4fb9-b0f8-ade285c5f42c None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:56.455150 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d98d24b-cfd3-4fb9-b0f8-ade285c5f42c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:56.455454 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d98d24b-cfd3-4fb9-b0f8-ade285c5f42c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:56.457115 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d98d24b-cfd3-4fb9-b0f8-ade285c5f42c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:56.471958 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 387/757] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:56.476994 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98d24b-cfd3-4fb9-b0f8-ade285c5f42c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:56.508161 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63956d62-40db-45cb-9032-f61676ca243f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:56.508690 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63956d62-40db-45cb-9032-f61676ca243f None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:56.508967 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63956d62-40db-45cb-9032-f61676ca243f None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:56.509238 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63956d62-40db-45cb-9032-f61676ca243f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:56.632865 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98d24b-cfd3-4fb9-b0f8-ade285c5f42c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:56.636064 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:56.637939 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63956d62-40db-45cb-9032-f61676ca243f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:56.640061 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63956d62-40db-45cb-9032-f61676ca243f None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:56.705047 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:56.709281 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e459ca0e-9093-49a3-88ae-e1d6ff2a2f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:56.709926 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459ca0e-9093-49a3-88ae-e1d6ff2a2f6b None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:56.710203 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459ca0e-9093-49a3-88ae-e1d6ff2a2f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:56.710474 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459ca0e-9093-49a3-88ae-e1d6ff2a2f6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:56.717877 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e459ca0e-9093-49a3-88ae-e1d6ff2a2f6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:56.719379 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e459ca0e-9093-49a3-88ae-e1d6ff2a2f6b None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:56.731178 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-e459ca0e-9093-49a3-88ae-e1d6ff2a2f6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:56.731178 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:56.731178 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:56.731178 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:56.731178 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:56.731178 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:56.731178 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:56.731178 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:56.731178 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:56.731178 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:56.731178 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:56.731178 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:56.731178 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:54:56.731178 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:54:56.731178 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:54:56.732142 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:54:56.732142 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:56.732142 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:56.732142 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:56.732142 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:56.732142 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:56.732142 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:56.732142 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:56.732142 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:56.732142 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:56.732142 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:56.732142 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:56.732142 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:56.732142 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:56.732142 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:56.732142 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:56.732142 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:56.733582 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:54:56.733582 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:54:56.733582 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:54:56.733582 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:54:56.733582 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:54:56.733582 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:54:56.733582 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:56.733582 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:56.733582 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 388/758] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:54:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:56.740258 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e459ca0e-9093-49a3-88ae-e1d6ff2a2f6b None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:56.791222 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 371/759] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:54:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 14:54:56.798314 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63956d62-40db-45cb-9032-f61676ca243f None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:56.922829 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63956d62-40db-45cb-9032-f61676ca243f None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:56.932796 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e459ca0e-9093-49a3-88ae-e1d6ff2a2f6b None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:56.963253 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a68bc-4840-4f42-a1ad-de115f00b7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:56.964291 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a68bc-4840-4f42-a1ad-de115f00b7a2 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:56.964829 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a68bc-4840-4f42-a1ad-de115f00b7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:56.965307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a68bc-4840-4f42-a1ad-de115f00b7a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:56.977385 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a68bc-4840-4f42-a1ad-de115f00b7a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:56.979179 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a68bc-4840-4f42-a1ad-de115f00b7a2 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:56.998140 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-562a68bc-4840-4f42-a1ad-de115f00b7a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:56.998140 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:56.998140 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:56.998140 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:56.998140 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:56.998140 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:56.998140 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:56.998140 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:56.998140 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:56.998140 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:56.998140 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:56.998140 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:56.998140 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:54:56.998140 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:54:56.998140 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:54:56.998140 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:54:56.999654 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:56.999654 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:56.999654 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:56.999654 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:56.999654 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:56.999654 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:56.999654 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:56.999654 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:56.999654 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:56.999654 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:56.999654 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:56.999654 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:56.999654 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:56.999654 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:56.999654 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:56.999654 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:56.999654 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:54:57.000557 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:54:57.000557 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:54:57.000557 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:57.002704 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 372/760] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:54:57.013705 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a68bc-4840-4f42-a1ad-de115f00b7a2 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:57.019276 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcc7be0-a90d-40af-8d62-fb87ac81dda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:57.020073 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc7be0-a90d-40af-8d62-fb87ac81dda8 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:57.020517 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc7be0-a90d-40af-8d62-fb87ac81dda8 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:57.020929 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc7be0-a90d-40af-8d62-fb87ac81dda8 None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:57.021771 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcc7be0-a90d-40af-8d62-fb87ac81dda8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:57.022213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcc7be0-a90d-40af-8d62-fb87ac81dda8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:57.023810 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcc7be0-a90d-40af-8d62-fb87ac81dda8 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:57.035708 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:57.038257 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 389/761] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:54:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:57.043597 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcc7be0-a90d-40af-8d62-fb87ac81dda8 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:57.075644 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:57.200603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcc7be0-a90d-40af-8d62-fb87ac81dda8 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:57.212515 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a68bc-4840-4f42-a1ad-de115f00b7a2 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:57.269475 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a33e8-cde0-442f-a392-b1046116597c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:57.270098 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a33e8-cde0-442f-a392-b1046116597c None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:57.270382 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a33e8-cde0-442f-a392-b1046116597c None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:57.270662 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a33e8-cde0-442f-a392-b1046116597c None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:57.271352 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461a33e8-cde0-442f-a392-b1046116597c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:57.271661 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461a33e8-cde0-442f-a392-b1046116597c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:57.273074 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461a33e8-cde0-442f-a392-b1046116597c None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:57.277962 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3750ef-0ce9-475e-8cd5-c9554b4867f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:57.278643 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3750ef-0ce9-475e-8cd5-c9554b4867f2 None admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:57.278952 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3750ef-0ce9-475e-8cd5-c9554b4867f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:57.279232 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3750ef-0ce9-475e-8cd5-c9554b4867f2 None admin] PATH_INFO: `/v3/projects/9e4a3d44808e4b4092cfc4ee284c0176/users/fc0ea93084b0482fb2dd6d802987942f/roles/99e8fe7df8e7446680a81d4eaaa1cb68` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:57.281742 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3750ef-0ce9-475e-8cd5-c9554b4867f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e4a3d44808e4b4092cfc4ee284c0176, user_id=fc0ea93084b0482fb2dd6d802987942f, role_id=99e8fe7df8e7446680a81d4eaaa1cb68)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:57.283947 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3750ef-0ce9-475e-8cd5-c9554b4867f2 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:57.285941 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 373/762] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:54:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:54:57.291740 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a33e8-cde0-442f-a392-b1046116597c None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:57.308062 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 390/763] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 14:54:57 2022] PUT /identity/v3/projects/9e4a3d44808e4b4092cfc4ee284c0176/users/fc0ea93084b0482fb2dd6d802987942f/roles/99e8fe7df8e7446680a81d4eaaa1cb68 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:54:57.381277 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:57.415730 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a33e8-cde0-442f-a392-b1046116597c None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:57.455140 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72baa23d-82f9-492c-9c94-5a74dbba5841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:57.455937 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72baa23d-82f9-492c-9c94-5a74dbba5841 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:57.456318 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72baa23d-82f9-492c-9c94-5a74dbba5841 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:57.456692 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72baa23d-82f9-492c-9c94-5a74dbba5841 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:57.463920 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72baa23d-82f9-492c-9c94-5a74dbba5841 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:57.465493 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72baa23d-82f9-492c-9c94-5a74dbba5841 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:57.478971 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-72baa23d-82f9-492c-9c94-5a74dbba5841 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:57.478971 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:57.478971 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:57.478971 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:57.478971 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:57.478971 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:57.478971 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:57.478971 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:57.478971 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:57.478971 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:57.478971 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:57.478971 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:57.478971 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:54:57.478971 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:54:57.478971 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:54:57.480085 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:54:57.480085 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:57.480085 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:57.480085 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:57.480085 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:57.480085 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:57.480085 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:57.480085 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:57.480085 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:57.480085 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:57.480085 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:57.480085 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:57.480085 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:57.480085 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:57.480085 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:57.480085 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:57.480085 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:57.481159 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:54:57.481159 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:54:57.481159 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:54:57.481159 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:54:57.481159 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:54:57.481159 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:54:57.481159 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:54:57.481159 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:57.483362 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 374/764] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:54:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:54:57.489744 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3750ef-0ce9-475e-8cd5-c9554b4867f2 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:57.627481 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:57.649546 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3750ef-0ce9-475e-8cd5-c9554b4867f2 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:57.690857 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d2313-cab1-482c-866e-a0ad566a6555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:57.691487 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d2313-cab1-482c-866e-a0ad566a6555 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:57.691769 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d2313-cab1-482c-866e-a0ad566a6555 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:57.692044 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d2313-cab1-482c-866e-a0ad566a6555 None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:57.692820 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186d2313-cab1-482c-866e-a0ad566a6555 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:57.693130 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186d2313-cab1-482c-866e-a0ad566a6555 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:57.694624 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186d2313-cab1-482c-866e-a0ad566a6555 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:57.708371 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 391/765] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:54:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:57.715209 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72baa23d-82f9-492c-9c94-5a74dbba5841 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:57.850290 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72baa23d-82f9-492c-9c94-5a74dbba5841 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:57.859455 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d78642-18db-47cc-8480-f17aaa25bcb4 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:57.859766 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d78642-18db-47cc-8480-f17aaa25bcb4 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:57.860006 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d78642-18db-47cc-8480-f17aaa25bcb4 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:57.861571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 392/766] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:54:57.868042 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047421a-4ffd-46a1-afd8-71b6033a2e44 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:57.868323 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047421a-4ffd-46a1-afd8-71b6033a2e44 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:57.868600 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047421a-4ffd-46a1-afd8-71b6033a2e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:57.908893 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-2047421a-4ffd-46a1-afd8-71b6033a2e44 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:57.909493 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:57.910329 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-2047421a-4ffd-46a1-afd8-71b6033a2e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:57.917292 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d3e52b-702e-4bd3-86cc-842cc1594cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:57.917912 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3e52b-702e-4bd3-86cc-842cc1594cf2 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:57.918193 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3e52b-702e-4bd3-86cc-842cc1594cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:57.918464 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3e52b-702e-4bd3-86cc-842cc1594cf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:57.919306 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d3e52b-702e-4bd3-86cc-842cc1594cf2 None admin] RBAC: Adding query filter params (role.id=533fa341c4f64cd4b0a7ebb668f9106a, scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176, user.id=0aafb2bbba1d466fb22bb0824a531ddc) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:57.919617 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d3e52b-702e-4bd3-86cc-842cc1594cf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:57.921098 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d3e52b-702e-4bd3-86cc-842cc1594cf2 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:57.935456 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 375/767] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:54:57 2022] GET /identity/v3/role_assignments?user.id=0aafb2bbba1d466fb22bb0824a531ddc&scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176&role.id=533fa341c4f64cd4b0a7ebb668f9106a => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:58.122774 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 393/768] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 14:54:58.385164 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed316e-8221-4932-81dc-693a520c68bd None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:58.385718 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed316e-8221-4932-81dc-693a520c68bd None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:58.385718 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed316e-8221-4932-81dc-693a520c68bd None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:58.387343 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 376/769] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:58.404911 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f092066-4e45-4006-b292-89f5eaf9a475 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:58.405407 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f092066-4e45-4006-b292-89f5eaf9a475 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:58.405630 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f092066-4e45-4006-b292-89f5eaf9a475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:58.428636 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-2f092066-4e45-4006-b292-89f5eaf9a475 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:58.429839 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-2f092066-4e45-4006-b292-89f5eaf9a475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:58.521337 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 394/770] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:54:58.541940 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed316e-8221-4932-81dc-693a520c68bd None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:58.689206 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed316e-8221-4932-81dc-693a520c68bd None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:58.738286 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126217f-a959-47eb-b622-2ef4242b14e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:58.738959 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126217f-a959-47eb-b622-2ef4242b14e2 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:58.739244 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126217f-a959-47eb-b622-2ef4242b14e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:58.739520 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126217f-a959-47eb-b622-2ef4242b14e2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:58.740107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c126217f-a959-47eb-b622-2ef4242b14e2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:58.741488 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c126217f-a959-47eb-b622-2ef4242b14e2 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:58.754601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-c126217f-a959-47eb-b622-2ef4242b14e2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 14:54:58.754601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:58.754601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:54:58.754601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:54:58.754601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:54:58.754601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:54:58.754601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:54:58.754601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:54:58.754601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 14:54:58.754601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:58.754601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:54:58.754601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:58.754601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:58.754601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:58.754601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:58.754601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:58.754601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:58.755619 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:58.755619 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:58.755619 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:58.755619 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:58.755619 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:58.755619 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:58.755619 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:54:58.755619 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:54:58.755619 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:54:58.755619 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:54:58.755619 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:58.755619 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:58.755619 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:58.755619 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:58.755619 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:58.755619 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:58.757005 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:58.757005 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:58.757005 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:58.757005 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:58.757005 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:58.757005 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:58.757005 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:58.757005 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:58.757005 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:58.757005 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:58.757005 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:54:58.757005 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:54:58.757005 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 14:54:58.757005 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:58.757005 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 377/771] 10.222.0.33 () {56 vars in 1129 bytes} [Sun Sep 11 14:54:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:58.761611 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f092066-4e45-4006-b292-89f5eaf9a475 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:58.768877 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:58.824833 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126217f-a959-47eb-b622-2ef4242b14e2 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:58.916267 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:58.947793 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f092066-4e45-4006-b292-89f5eaf9a475 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:58.967686 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126217f-a959-47eb-b622-2ef4242b14e2 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:59.004110 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed685c-f965-4da6-8166-339087466a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:59.004110 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed685c-f965-4da6-8166-339087466a14 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:59.004110 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed685c-f965-4da6-8166-339087466a14 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:59.004453 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed685c-f965-4da6-8166-339087466a14 None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:59.004848 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed685c-f965-4da6-8166-339087466a14 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:59.005155 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed685c-f965-4da6-8166-339087466a14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:59.006559 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed685c-f965-4da6-8166-339087466a14 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:59.012101 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 395/772] 10.222.0.33 () {56 vars in 1124 bytes} [Sun Sep 11 14:54:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:59.013052 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ff6b8-881b-46e2-b06a-5b86f9e0b95c None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:59.013360 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ff6b8-881b-46e2-b06a-5b86f9e0b95c None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:59.013659 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ff6b8-881b-46e2-b06a-5b86f9e0b95c None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:59.021219 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 396/773] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:54:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:54:59.021219 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ff6b8-881b-46e2-b06a-5b86f9e0b95c None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:59.026352 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5e1e6-8004-4508-a870-15910f156b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:59.027203 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5e1e6-8004-4508-a870-15910f156b46 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:59.027863 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5e1e6-8004-4508-a870-15910f156b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:59.028377 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5e1e6-8004-4508-a870-15910f156b46 None admin] PATH_INFO: `/v3/services/network` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:59.028975 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5e1e6-8004-4508-a870-15910f156b46 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:59.031123 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5e1e6-8004-4508-a870-15910f156b46 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:59.040579 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-ebf5e1e6-8004-4508-a870-15910f156b46 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 14:54:59.040579 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:59.040579 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:54:59.040579 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:54:59.040579 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:54:59.040579 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:54:59.040579 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:54:59.040579 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:54:59.040579 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 14:54:59.040579 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:59.040579 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:54:59.040579 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:59.040579 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:59.040579 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:59.040579 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:59.040579 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:59.040579 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:59.041587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:59.041587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:59.041587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:59.041587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:59.041587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:59.041587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:59.041587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:54:59.041587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:54:59.041587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:54:59.041587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:54:59.041587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:59.041587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:59.041587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:59.041587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:59.041587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:59.041587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:59.042839 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:59.042839 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:59.042839 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:59.042839 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:59.042839 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:59.042839 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:59.042839 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:59.042839 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:59.042839 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:59.042839 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:59.042839 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:54:59.042839 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:54:59.042839 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 14:54:59.042839 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:54:59.042839 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 378/774] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:54:58 2022] GET /identity/v3/services/network => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:54:59.047369 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df504dc6-355f-4c44-8550-695d157bb831 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:59.047658 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df504dc6-355f-4c44-8550-695d157bb831 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:59.047943 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df504dc6-355f-4c44-8550-695d157bb831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:59.067784 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-df504dc6-355f-4c44-8550-695d157bb831 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:54:59.068995 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-df504dc6-355f-4c44-8550-695d157bb831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:54:59.118783 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:59.150837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 379/775] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:54:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:54:59.150837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df504dc6-355f-4c44-8550-695d157bb831 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:59.188286 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ff6b8-881b-46e2-b06a-5b86f9e0b95c None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:59.241140 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:59.251210 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc9bde-c42d-4055-9380-8fc117d47c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:59.251820 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc9bde-c42d-4055-9380-8fc117d47c4b None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:59.252150 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc9bde-c42d-4055-9380-8fc117d47c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:59.252431 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc9bde-c42d-4055-9380-8fc117d47c4b None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:59.253062 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc9bde-c42d-4055-9380-8fc117d47c4b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:59.253361 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc9bde-c42d-4055-9380-8fc117d47c4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:59.254739 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc9bde-c42d-4055-9380-8fc117d47c4b None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:59.259789 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 397/776] 10.222.0.33 () {56 vars in 1124 bytes} [Sun Sep 11 14:54:59 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:59.266007 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc9bde-c42d-4055-9380-8fc117d47c4b None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:59.316656 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df504dc6-355f-4c44-8550-695d157bb831 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:59.383244 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966c88a-5c44-4b88-b851-ab3a0fc45b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:59.383871 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966c88a-5c44-4b88-b851-ab3a0fc45b32 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:59.384154 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966c88a-5c44-4b88-b851-ab3a0fc45b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:59.384426 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966c88a-5c44-4b88-b851-ab3a0fc45b32 None admin] PATH_INFO: `/v3/services` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:59.385067 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7966c88a-5c44-4b88-b851-ab3a0fc45b32 None admin] RBAC: Adding query filter params (name=network) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:59.385370 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7966c88a-5c44-4b88-b851-ab3a0fc45b32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:59.386756 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7966c88a-5c44-4b88-b851-ab3a0fc45b32 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:59.400533 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 380/777] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 11 14:54:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:59.406874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966c88a-5c44-4b88-b851-ab3a0fc45b32 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:59.457138 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc9bde-c42d-4055-9380-8fc117d47c4b None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:59.510253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:59.517613 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0158322f-490a-4dd2-82f8-634099f3d58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:59.518543 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158322f-490a-4dd2-82f8-634099f3d58a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:59.519061 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158322f-490a-4dd2-82f8-634099f3d58a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:59.519531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158322f-490a-4dd2-82f8-634099f3d58a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:59.521245 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0158322f-490a-4dd2-82f8-634099f3d58a None admin] RBAC: Adding query filter params (interface=public, service_id=606fc65a406647cc9513a45f7648aa9d, region_id=RegionOne) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:59.521978 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0158322f-490a-4dd2-82f8-634099f3d58a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:59.523925 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0158322f-490a-4dd2-82f8-634099f3d58a None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:59.533956 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 398/778] 10.222.0.33 () {56 vars in 1251 bytes} [Sun Sep 11 14:54:59 2022] GET /identity/v3/endpoints?service_id=606fc65a406647cc9513a45f7648aa9d&interface=public®ion_id=RegionOne => generated 194 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:59.535472 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0158322f-490a-4dd2-82f8-634099f3d58a None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:59.567324 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966c88a-5c44-4b88-b851-ab3a0fc45b32 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:59.587115 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:59.608515 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab4392-31c5-4375-958f-5a9aa7c38f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:59.609144 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab4392-31c5-4375-958f-5a9aa7c38f29 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:59.609414 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab4392-31c5-4375-958f-5a9aa7c38f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:59.609690 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab4392-31c5-4375-958f-5a9aa7c38f29 None admin] PATH_INFO: `/v3/services` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:59.610326 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faab4392-31c5-4375-958f-5a9aa7c38f29 None admin] RBAC: Adding query filter params (type=network) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:59.610631 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faab4392-31c5-4375-958f-5a9aa7c38f29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:59.612043 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faab4392-31c5-4375-958f-5a9aa7c38f29 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:59.619255 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 381/779] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 11 14:54:59 2022] GET /identity/v3/services?type=network => generated 350 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:59.620064 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab4392-31c5-4375-958f-5a9aa7c38f29 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:59.655848 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0158322f-490a-4dd2-82f8-634099f3d58a None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:59.693978 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109a894-c109-4988-91a8-46bd3a3b2577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:59.694990 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109a894-c109-4988-91a8-46bd3a3b2577 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:59.695466 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109a894-c109-4988-91a8-46bd3a3b2577 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:59.696021 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109a894-c109-4988-91a8-46bd3a3b2577 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:59.707817 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b109a894-c109-4988-91a8-46bd3a3b2577 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:54:59.713301 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-b109a894-c109-4988-91a8-46bd3a3b2577 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:54:59.713301 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:54:59.713301 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:54:59.713301 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:54:59.713301 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:54:59.713301 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:54:59.713301 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:54:59.713301 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:54:59.713301 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:54:59.713301 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:54:59.713301 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:54:59.713301 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:54:59.713301 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:54:59.713301 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:54:59.713301 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:54:59.713301 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:54:59.714232 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:54:59.714232 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:54:59.714232 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:54:59.714232 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:54:59.714232 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:54:59.714232 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:54:59.714232 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:54:59.714232 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:54:59.714232 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:54:59.714232 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:54:59.714232 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:54:59.714232 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:54:59.714232 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:54:59.714232 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:54:59.714232 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:54:59.714232 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:54:59.715106 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:54:59.715106 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:54:59.715106 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:54:59.715106 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:54:59.715106 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:54:59.715106 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:54:59.715106 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:54:59.715106 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:54:59.715106 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:54:59.715106 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:54:59.715106 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:54:59.715106 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:54:59.722784 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 399/780] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:54:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:54:59.725243 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109a894-c109-4988-91a8-46bd3a3b2577 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:59.728328 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab4392-31c5-4375-958f-5a9aa7c38f29 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:59.767177 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4c159-4968-45d9-a1a7-3b920c8684d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:59.767699 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4c159-4968-45d9-a1a7-3b920c8684d2 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:59.767979 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4c159-4968-45d9-a1a7-3b920c8684d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:59.768255 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4c159-4968-45d9-a1a7-3b920c8684d2 None admin] PATH_INFO: `/v3/services` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:59.768863 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4c159-4968-45d9-a1a7-3b920c8684d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:59.770248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4c159-4968-45d9-a1a7-3b920c8684d2 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:59.775878 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 382/781] 10.222.0.33 () {56 vars in 1087 bytes} [Sun Sep 11 14:54:59 2022] GET /identity/v3/services => generated 772 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:59.776675 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4c159-4968-45d9-a1a7-3b920c8684d2 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:59.791962 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:59.833631 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:54:59.849202 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109a894-c109-4988-91a8-46bd3a3b2577 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:59.889930 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ead4eb-2f27-43ea-a24a-5874770420de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:59.890980 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ead4eb-2f27-43ea-a24a-5874770420de None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:59.891486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ead4eb-2f27-43ea-a24a-5874770420de None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:59.891953 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ead4eb-2f27-43ea-a24a-5874770420de None admin] PATH_INFO: `/v3/endpoints` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:59.892878 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ead4eb-2f27-43ea-a24a-5874770420de None admin] RBAC: Adding query filter params (interface=public, service_id=c24c76eca3c542bca2dbc5c227d2fe11, region_id=RegionOne) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:59.893468 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ead4eb-2f27-43ea-a24a-5874770420de None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:59.895110 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ead4eb-2f27-43ea-a24a-5874770420de None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:59.901839 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 400/782] 10.222.0.33 () {56 vars in 1251 bytes} [Sun Sep 11 14:54:59 2022] GET /identity/v3/endpoints?service_id=c24c76eca3c542bca2dbc5c227d2fe11&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:59.909768 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ead4eb-2f27-43ea-a24a-5874770420de None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:54:59.909993 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4c159-4968-45d9-a1a7-3b920c8684d2 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:54:59.948698 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7219aaa5-0f9a-4016-befd-8b691fccd0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:54:59.949332 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219aaa5-0f9a-4016-befd-8b691fccd0ef None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:54:59.949613 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219aaa5-0f9a-4016-befd-8b691fccd0ef None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:54:59.949889 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219aaa5-0f9a-4016-befd-8b691fccd0ef None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:54:59.950567 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7219aaa5-0f9a-4016-befd-8b691fccd0ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:54:59.950915 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7219aaa5-0f9a-4016-befd-8b691fccd0ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:54:59.952906 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7219aaa5-0f9a-4016-befd-8b691fccd0ef None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:54:59.962779 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 383/783] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:54:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:54:59.969383 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7219aaa5-0f9a-4016-befd-8b691fccd0ef None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:00.125547 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ead4eb-2f27-43ea-a24a-5874770420de None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:00.135892 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7219aaa5-0f9a-4016-befd-8b691fccd0ef None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:00.145427 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:00.163800 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:00.185128 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45f3ba2-cbb1-410a-8454-b9bc82f94fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:00.186295 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f3ba2-cbb1-410a-8454-b9bc82f94fd2 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:00.186794 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f3ba2-cbb1-410a-8454-b9bc82f94fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:00.187322 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f3ba2-cbb1-410a-8454-b9bc82f94fd2 None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:00.187944 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45f3ba2-cbb1-410a-8454-b9bc82f94fd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:00.189339 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45f3ba2-cbb1-410a-8454-b9bc82f94fd2 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:00.194753 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 401/784] 10.222.0.33 () {56 vars in 1087 bytes} [Sun Sep 11 14:54:59 2022] GET /identity/v3/services => generated 772 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:00.196023 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984abea7-2301-4c4e-85ae-9a54a1e6640f None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:00.196308 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984abea7-2301-4c4e-85ae-9a54a1e6640f None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:00.196582 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984abea7-2301-4c4e-85ae-9a54a1e6640f None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:00.198032 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 402/785] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:00.199166 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a47dc-ea42-4880-963f-131afa68a029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:00.199979 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a47dc-ea42-4880-963f-131afa68a029 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:00.200361 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a47dc-ea42-4880-963f-131afa68a029 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:00.200735 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a47dc-ea42-4880-963f-131afa68a029 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:00.204381 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec6b0e-fd4b-4caf-ae75-b673c9de80ee None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:00.204659 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec6b0e-fd4b-4caf-ae75-b673c9de80ee None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:00.205434 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec6b0e-fd4b-4caf-ae75-b673c9de80ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:00.216001 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a47dc-ea42-4880-963f-131afa68a029 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:00.218046 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a47dc-ea42-4880-963f-131afa68a029 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:00.240048 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-3aec6b0e-fd4b-4caf-ae75-b673c9de80ee None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:00.241298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-3aec6b0e-fd4b-4caf-ae75-b673c9de80ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:00.250449 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-d50a47dc-ea42-4880-963f-131afa68a029 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:00.250449 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:00.250449 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:00.250449 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:00.250449 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:00.250449 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:00.250449 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:00.250449 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:00.250449 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:00.250449 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:00.250449 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:00.250449 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:00.250449 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:55:00.250449 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:55:00.250449 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:55:00.250449 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:55:00.251515 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:00.251515 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:00.251515 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:00.251515 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:00.251515 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:00.251515 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:00.251515 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:00.251515 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:00.251515 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:00.251515 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:00.251515 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:00.251515 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:00.251515 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:00.251515 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:00.251515 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:00.251515 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:00.251515 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:55:00.252415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:55:00.252415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:00.252415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:00.252415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 384/786] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:54:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:00.264513 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a47dc-ea42-4880-963f-131afa68a029 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:00.390871 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 403/787] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 14:55:00.429965 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a47dc-ea42-4880-963f-131afa68a029 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:00.470568 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027525eb-297e-4101-a2a9-5ce38f53ed05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:00.471233 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027525eb-297e-4101-a2a9-5ce38f53ed05 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:00.471511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027525eb-297e-4101-a2a9-5ce38f53ed05 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:00.471785 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027525eb-297e-4101-a2a9-5ce38f53ed05 None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:00.472442 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027525eb-297e-4101-a2a9-5ce38f53ed05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:00.472746 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027525eb-297e-4101-a2a9-5ce38f53ed05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:00.474279 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027525eb-297e-4101-a2a9-5ce38f53ed05 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:00.493252 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 385/788] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:55:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:00.503924 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec6b0e-fd4b-4caf-ae75-b673c9de80ee None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:00.572252 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a82985-d4b8-469b-8cf1-bc3bcc0a718f None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:00.572566 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a82985-d4b8-469b-8cf1-bc3bcc0a718f None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:00.572802 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a82985-d4b8-469b-8cf1-bc3bcc0a718f None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:00.574590 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 386/789] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:55:00.579359 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778653a-9e55-49db-a73b-4e86c0272052 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:00.579637 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778653a-9e55-49db-a73b-4e86c0272052 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:00.579910 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778653a-9e55-49db-a73b-4e86c0272052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:00.612628 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-f778653a-9e55-49db-a73b-4e86c0272052 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:00.613840 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-f778653a-9e55-49db-a73b-4e86c0272052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:00.637830 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec6b0e-fd4b-4caf-ae75-b673c9de80ee None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:00.643204 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:00.666753 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:00.712135 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbd072b-3d4f-4f1b-9289-9d7cb68aacfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:00.713197 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd072b-3d4f-4f1b-9289-9d7cb68aacfa None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:00.714585 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd072b-3d4f-4f1b-9289-9d7cb68aacfa None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:00.715084 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd072b-3d4f-4f1b-9289-9d7cb68aacfa None admin] PATH_INFO: `/v3/users/swift` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:00.729638 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbd072b-3d4f-4f1b-9289-9d7cb68aacfa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:00.731467 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbd072b-3d4f-4f1b-9289-9d7cb68aacfa None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:00.751490 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-fcbd072b-3d4f-4f1b-9289-9d7cb68aacfa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 14:55:00.751490 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:00.751490 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:00.751490 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:00.751490 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:00.751490 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:00.751490 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:00.751490 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:00.751490 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:00.751490 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:00.751490 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:00.751490 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:00.751490 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:55:00.751490 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:55:00.751490 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:55:00.751490 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:55:00.752486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:00.752486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:00.752486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:55:00.752486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:55:00.752486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:55:00.752486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:55:00.752486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:00.752486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:00.752486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:00.752486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:00.752486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:00.752486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:00.752486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:00.752486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:00.752486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:00.752486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:00.752486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:00.752486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:00.753429 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:00.753429 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:00.753429 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:55:00.753429 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:55:00.753429 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:55:00.753429 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:55:00.753429 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:55:00.753429 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:55:00.753429 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 14:55:00.753429 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:00.753960 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 387/790] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 14:55:00.753960 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f778653a-9e55-49db-a73b-4e86c0272052 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:00.756054 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 404/791] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:55:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:00.767793 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbd072b-3d4f-4f1b-9289-9d7cb68aacfa None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:00.945929 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f778653a-9e55-49db-a73b-4e86c0272052 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:00.984409 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbd072b-3d4f-4f1b-9289-9d7cb68aacfa None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:01.031218 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc4b26-e403-424b-9d5a-bc73dae5c290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:01.032259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc4b26-e403-424b-9d5a-bc73dae5c290 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:01.032928 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc4b26-e403-424b-9d5a-bc73dae5c290 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:01.034436 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc4b26-e403-424b-9d5a-bc73dae5c290 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:01.055516 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffc4b26-e403-424b-9d5a-bc73dae5c290 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:55:01.060130 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8fb6f-d06c-4772-86d8-6061eb28a145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:01.061786 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8fb6f-d06c-4772-86d8-6061eb28a145 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:01.062276 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8fb6f-d06c-4772-86d8-6061eb28a145 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:01.062746 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8fb6f-d06c-4772-86d8-6061eb28a145 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:01.063735 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8fb6f-d06c-4772-86d8-6061eb28a145 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:01.066124 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-5ffc4b26-e403-424b-9d5a-bc73dae5c290 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:55:01.066124 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:01.066124 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:01.066124 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:01.066124 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:01.066124 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:01.066124 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:01.066124 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:01.066124 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:01.066124 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:01.066124 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:01.066124 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:01.066124 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:55:01.066124 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:55:01.066124 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:55:01.066124 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:55:01.067108 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:55:01.067108 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:55:01.067108 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:55:01.067108 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:55:01.067108 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:01.067108 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:01.067108 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:01.067108 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:01.067108 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:01.067108 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:01.067108 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:01.067108 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:01.067108 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:01.067108 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:01.067108 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:01.067108 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:01.067972 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:01.067972 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:01.067972 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:01.067972 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:01.067972 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:55:01.067972 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:55:01.067972 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:55:01.067972 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:55:01.067972 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:55:01.067972 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:55:01.067972 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:55:01.067972 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:01.067972 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 388/792] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:55:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:01.068692 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc4b26-e403-424b-9d5a-bc73dae5c290 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:01.069459 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8fb6f-d06c-4772-86d8-6061eb28a145 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:01.070268 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8fb6f-d06c-4772-86d8-6061eb28a145 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:01.087847 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:01.094385 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 405/793] 10.222.0.33 () {56 vars in 1135 bytes} [Sun Sep 11 14:55:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:01.095374 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8fb6f-d06c-4772-86d8-6061eb28a145 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:01.133629 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:01.214249 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc4b26-e403-424b-9d5a-bc73dae5c290 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:01.225633 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8fb6f-d06c-4772-86d8-6061eb28a145 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:01.258454 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05a66a-3453-4cbe-8b8d-965f9c3cfee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:01.259495 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05a66a-3453-4cbe-8b8d-965f9c3cfee5 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:01.260017 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05a66a-3453-4cbe-8b8d-965f9c3cfee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:01.260433 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05a66a-3453-4cbe-8b8d-965f9c3cfee5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:01.270712 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336270ce-fc58-4e64-b814-7742e885a2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:01.271471 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336270ce-fc58-4e64-b814-7742e885a2eb None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:01.271906 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336270ce-fc58-4e64-b814-7742e885a2eb None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:01.272283 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336270ce-fc58-4e64-b814-7742e885a2eb None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:01.273386 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336270ce-fc58-4e64-b814-7742e885a2eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:01.273798 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336270ce-fc58-4e64-b814-7742e885a2eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:01.274013 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe05a66a-3453-4cbe-8b8d-965f9c3cfee5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:55:01.275406 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336270ce-fc58-4e64-b814-7742e885a2eb None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:01.280664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-fe05a66a-3453-4cbe-8b8d-965f9c3cfee5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:55:01.280664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:01.280664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:01.280664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:01.280664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:01.280664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:01.280664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:01.280664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:01.280664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:01.280664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:01.280664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:01.280664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:01.280664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:55:01.280664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:55:01.280664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:55:01.280664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:55:01.281780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:55:01.281780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:55:01.281780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:55:01.281780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:55:01.281780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:01.281780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:01.281780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:01.281780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:01.281780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:01.281780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:01.281780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:01.281780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:01.281780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:01.281780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:01.281780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:01.281780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:01.282837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:01.282837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:01.282837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:01.282837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:01.282837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:55:01.282837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:55:01.282837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:55:01.282837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:55:01.282837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:55:01.282837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:55:01.282837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:55:01.282837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:01.285271 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 389/794] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 14:55:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:01.286203 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05a66a-3453-4cbe-8b8d-965f9c3cfee5 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:01.293606 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 406/795] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:55:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:01.294562 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336270ce-fc58-4e64-b814-7742e885a2eb None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:01.371151 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:01.413240 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05a66a-3453-4cbe-8b8d-965f9c3cfee5 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:01.422772 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:01.432310 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336270ce-fc58-4e64-b814-7742e885a2eb None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:01.460516 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380680e-1664-4dd8-a565-e0e3b4ebf7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:01.461322 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380680e-1664-4dd8-a565-e0e3b4ebf7ba None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:01.461731 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380680e-1664-4dd8-a565-e0e3b4ebf7ba None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:01.462328 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380680e-1664-4dd8-a565-e0e3b4ebf7ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:01.474506 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380680e-1664-4dd8-a565-e0e3b4ebf7ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:01.475913 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c135f6-3801-4a8e-b456-36a22c226980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:01.476340 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380680e-1664-4dd8-a565-e0e3b4ebf7ba None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:01.476921 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c135f6-3801-4a8e-b456-36a22c226980 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:01.477386 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c135f6-3801-4a8e-b456-36a22c226980 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:01.477852 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c135f6-3801-4a8e-b456-36a22c226980 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:01.478686 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c135f6-3801-4a8e-b456-36a22c226980 None admin] RBAC: Adding query filter params (name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:01.479189 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c135f6-3801-4a8e-b456-36a22c226980 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:01.480752 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c135f6-3801-4a8e-b456-36a22c226980 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:01.490448 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 407/796] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 14:55:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:01.491560 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c135f6-3801-4a8e-b456-36a22c226980 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:01.496983 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-9380680e-1664-4dd8-a565-e0e3b4ebf7ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:01.496983 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:01.496983 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:01.496983 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:01.496983 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:01.496983 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:01.496983 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:01.496983 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:01.496983 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:01.496983 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:01.496983 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:01.496983 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:01.496983 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:55:01.496983 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:55:01.496983 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:55:01.496983 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:55:01.497955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:01.497955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:01.497955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:01.497955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:01.497955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:01.497955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:01.497955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:01.497955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:01.497955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:01.497955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:01.497955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:01.497955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:01.497955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:01.497955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:01.497955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:01.497955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:01.497955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:55:01.499988 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:55:01.499988 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:01.499988 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:01.502938 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 390/797] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:55:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:01.503963 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380680e-1664-4dd8-a565-e0e3b4ebf7ba None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:01.619196 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c135f6-3801-4a8e-b456-36a22c226980 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:01.622159 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380680e-1664-4dd8-a565-e0e3b4ebf7ba None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:01.648232 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:01.661265 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e605a450-ce9d-4fb5-8fb1-9916cb6c6f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:01.661995 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605a450-ce9d-4fb5-8fb1-9916cb6c6f14 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:01.662348 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605a450-ce9d-4fb5-8fb1-9916cb6c6f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:01.662770 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605a450-ce9d-4fb5-8fb1-9916cb6c6f14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:01.668345 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0458b2-0adc-4e93-87a0-f3113be96015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:01.669251 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0458b2-0adc-4e93-87a0-f3113be96015 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:01.669729 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0458b2-0adc-4e93-87a0-f3113be96015 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:01.670669 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0458b2-0adc-4e93-87a0-f3113be96015 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:01.674099 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e605a450-ce9d-4fb5-8fb1-9916cb6c6f14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:01.675876 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e605a450-ce9d-4fb5-8fb1-9916cb6c6f14 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:01.684719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0458b2-0adc-4e93-87a0-f3113be96015 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:01.687107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0458b2-0adc-4e93-87a0-f3113be96015 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:01.691958 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-e605a450-ce9d-4fb5-8fb1-9916cb6c6f14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:01.691958 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:01.691958 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:01.691958 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:01.691958 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:01.691958 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:01.691958 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:01.691958 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:01.691958 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:01.691958 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:01.691958 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:01.691958 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:01.691958 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:55:01.691958 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:55:01.691958 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:55:01.691958 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:55:01.692895 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:01.692895 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:01.692895 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:01.692895 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:01.692895 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:01.692895 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:01.692895 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:01.692895 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:01.692895 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:01.692895 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:01.692895 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:01.692895 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:01.692895 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:01.692895 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:01.692895 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:01.692895 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:01.692895 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:55:01.693817 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:55:01.693817 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:01.693817 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:01.693817 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 408/798] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:55:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:01.693817 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e605a450-ce9d-4fb5-8fb1-9916cb6c6f14 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:01.710531 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-8b0458b2-0adc-4e93-87a0-f3113be96015 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:01.710531 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:01.710531 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:01.710531 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:01.710531 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:01.710531 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:01.710531 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:01.710531 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:01.710531 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:01.710531 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:01.710531 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:01.710531 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:01.710531 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:55:01.710531 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:55:01.710531 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:55:01.710531 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:55:01.711712 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:01.711712 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:01.711712 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:01.711712 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:01.711712 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:01.711712 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:01.711712 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:01.711712 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:01.711712 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:01.711712 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:01.711712 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:01.711712 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:01.711712 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:01.711712 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:01.711712 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:01.711712 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:01.711712 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:55:01.712633 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:55:01.712633 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:01.712633 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:01.717184 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 391/799] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:55:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:01.717851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:01.719152 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0458b2-0adc-4e93-87a0-f3113be96015 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:01.820889 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e605a450-ce9d-4fb5-8fb1-9916cb6c6f14 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:01.853301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0458b2-0adc-4e93-87a0-f3113be96015 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:01.856618 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb4ba3-4f1a-429f-9ff7-d25e3a44e96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:01.857217 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb4ba3-4f1a-429f-9ff7-d25e3a44e96e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:01.857525 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb4ba3-4f1a-429f-9ff7-d25e3a44e96e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:01.857877 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb4ba3-4f1a-429f-9ff7-d25e3a44e96e None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:01.858547 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb4ba3-4f1a-429f-9ff7-d25e3a44e96e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:01.858893 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb4ba3-4f1a-429f-9ff7-d25e3a44e96e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:01.860286 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb4ba3-4f1a-429f-9ff7-d25e3a44e96e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:01.877563 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 409/800] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:55:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:01.877563 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb4ba3-4f1a-429f-9ff7-d25e3a44e96e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:01.907068 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99fabcb-276b-4446-ba65-beebbcdaf613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:01.908298 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fabcb-276b-4446-ba65-beebbcdaf613 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:01.909203 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fabcb-276b-4446-ba65-beebbcdaf613 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:01.909956 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fabcb-276b-4446-ba65-beebbcdaf613 None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:01.910827 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99fabcb-276b-4446-ba65-beebbcdaf613 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:01.911337 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99fabcb-276b-4446-ba65-beebbcdaf613 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:01.914048 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99fabcb-276b-4446-ba65-beebbcdaf613 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:01.930856 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 392/801] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:55:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:01.932090 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99fabcb-276b-4446-ba65-beebbcdaf613 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:01.952688 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:01.980088 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:02.009654 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb4ba3-4f1a-429f-9ff7-d25e3a44e96e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:02.061903 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284489b-388b-4883-9f85-77c44ae018ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:02.062956 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284489b-388b-4883-9f85-77c44ae018ee None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:02.063276 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99fabcb-276b-4446-ba65-beebbcdaf613 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:02.063661 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284489b-388b-4883-9f85-77c44ae018ee None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:02.064186 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284489b-388b-4883-9f85-77c44ae018ee None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:02.065401 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7284489b-388b-4883-9f85-77c44ae018ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:02.065923 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7284489b-388b-4883-9f85-77c44ae018ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:02.068621 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7284489b-388b-4883-9f85-77c44ae018ee None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:02.090934 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 410/802] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:55:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:02.092514 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284489b-388b-4883-9f85-77c44ae018ee None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:02.109005 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd1afdb-cc43-4ad5-a3fc-071c40f76275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:02.109631 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd1afdb-cc43-4ad5-a3fc-071c40f76275 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:02.109983 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd1afdb-cc43-4ad5-a3fc-071c40f76275 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:02.110329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd1afdb-cc43-4ad5-a3fc-071c40f76275 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:02.117594 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd1afdb-cc43-4ad5-a3fc-071c40f76275 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:02.119158 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd1afdb-cc43-4ad5-a3fc-071c40f76275 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:02.131328 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-7dd1afdb-cc43-4ad5-a3fc-071c40f76275 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:55:02.131328 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:02.131328 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:02.131328 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:02.131328 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:02.131328 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:02.131328 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:02.131328 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:02.131328 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:02.131328 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:02.131328 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:02.131328 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:02.131328 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:55:02.131328 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:55:02.131328 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:55:02.132264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:55:02.132264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:02.132264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:02.132264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:02.132264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:02.132264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:02.132264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:02.132264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:02.132264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:02.132264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:02.132264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:02.132264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:02.132264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:02.132264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:02.132264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:02.132264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:02.132264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:02.133174 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:55:02.133174 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:55:02.133174 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:55:02.133174 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:55:02.133174 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:55:02.133174 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:55:02.133174 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:55:02.133174 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:02.133174 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 393/803] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:55:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:02.133665 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd1afdb-cc43-4ad5-a3fc-071c40f76275 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:02.226292 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284489b-388b-4883-9f85-77c44ae018ee None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:02.247335 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:02.256922 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd1afdb-cc43-4ad5-a3fc-071c40f76275 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:02.257091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:02.267824 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f11311-1cb4-41be-986e-9e99a10ecb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:02.269019 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f11311-1cb4-41be-986e-9e99a10ecb42 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:02.269724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f11311-1cb4-41be-986e-9e99a10ecb42 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:02.270182 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f11311-1cb4-41be-986e-9e99a10ecb42 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:02.280375 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f11311-1cb4-41be-986e-9e99a10ecb42 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:02.282860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f11311-1cb4-41be-986e-9e99a10ecb42 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:02.295399 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee484ca5-323c-44c9-a7ea-eabb326fe317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:02.295987 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-15f11311-1cb4-41be-986e-9e99a10ecb42 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 14:55:02.295987 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:02.295987 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:02.295987 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:02.295987 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:02.295987 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:02.295987 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:02.295987 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:02.295987 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:02.295987 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:02.295987 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:02.295987 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:02.295987 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:55:02.295987 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:55:02.295987 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:55:02.295987 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:55:02.296944 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:02.296944 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:02.296944 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:55:02.296944 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:55:02.296944 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:55:02.296944 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:55:02.296944 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:02.296944 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:02.296944 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:02.296944 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:02.296944 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:02.296944 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:02.296944 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:02.296944 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:02.296944 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:02.296944 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:02.296944 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:02.296944 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:02.297895 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:02.297895 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:02.297895 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:55:02.297895 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:55:02.297895 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:55:02.297895 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:55:02.297895 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:55:02.297895 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:55:02.297895 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 14:55:02.297895 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:02.298446 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee484ca5-323c-44c9-a7ea-eabb326fe317 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:02.298446 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee484ca5-323c-44c9-a7ea-eabb326fe317 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:02.298446 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee484ca5-323c-44c9-a7ea-eabb326fe317 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:02.298638 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 411/804] 10.222.0.33 () {56 vars in 1093 bytes} [Sun Sep 11 14:55:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:02.298638 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f11311-1cb4-41be-986e-9e99a10ecb42 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:02.306642 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee484ca5-323c-44c9-a7ea-eabb326fe317 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:02.308129 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee484ca5-323c-44c9-a7ea-eabb326fe317 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:02.320343 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-ee484ca5-323c-44c9-a7ea-eabb326fe317 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 14:55:02.320343 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:02.320343 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:02.320343 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:02.320343 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:02.320343 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:02.320343 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:02.320343 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:02.320343 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:02.320343 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:02.320343 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:02.320343 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:02.320343 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:55:02.320343 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:55:02.320343 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:55:02.320343 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:55:02.321296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:02.321296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:02.321296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:55:02.321296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:55:02.321296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:55:02.321296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:55:02.321296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:02.321296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:02.321296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:02.321296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:02.321296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:02.321296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:02.321296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:02.321296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:02.321296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:02.321296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:02.321296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:02.321296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:02.322266 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:02.322266 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:02.322266 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:55:02.322266 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:55:02.322266 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:55:02.322266 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:55:02.322266 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:55:02.322266 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:55:02.322266 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 14:55:02.322266 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:02.326097 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 394/805] 10.222.0.33 () {56 vars in 1117 bytes} [Sun Sep 11 14:55:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:02.327967 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9c2b3-28ca-42bf-b2f1-38788a2531fa None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:02.328345 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9c2b3-28ca-42bf-b2f1-38788a2531fa None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:02.328531 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9c2b3-28ca-42bf-b2f1-38788a2531fa None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:02.330861 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 395/806] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:02.330861 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9c2b3-28ca-42bf-b2f1-38788a2531fa None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:02.413362 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f11311-1cb4-41be-986e-9e99a10ecb42 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:02.442381 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9c2b3-28ca-42bf-b2f1-38788a2531fa None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:02.451615 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6d9d8-f58e-4721-bc67-79c9ac3c8d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:02.452205 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6d9d8-f58e-4721-bc67-79c9ac3c8d33 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:02.452421 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6d9d8-f58e-4721-bc67-79c9ac3c8d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:02.452698 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6d9d8-f58e-4721-bc67-79c9ac3c8d33 None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:02.453389 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb6d9d8-f58e-4721-bc67-79c9ac3c8d33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:02.453695 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb6d9d8-f58e-4721-bc67-79c9ac3c8d33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:02.455191 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb6d9d8-f58e-4721-bc67-79c9ac3c8d33 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:02.472723 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 412/807] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:55:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:02.472723 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6d9d8-f58e-4721-bc67-79c9ac3c8d33 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:02.486668 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83af1544-7881-40ca-9b68-c53e19ac57ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:02.487396 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af1544-7881-40ca-9b68-c53e19ac57ab None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:02.487605 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af1544-7881-40ca-9b68-c53e19ac57ab None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:02.487878 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af1544-7881-40ca-9b68-c53e19ac57ab None admin] PATH_INFO: `/v3/users` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:02.488603 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83af1544-7881-40ca-9b68-c53e19ac57ab None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:02.488908 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83af1544-7881-40ca-9b68-c53e19ac57ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:02.490362 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83af1544-7881-40ca-9b68-c53e19ac57ab None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:02.514592 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 396/808] 10.222.0.33 () {56 vars in 1133 bytes} [Sun Sep 11 14:55:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:02.514592 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acecd2-ecee-460c-9a12-812b58ce96af None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:02.514592 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acecd2-ecee-460c-9a12-812b58ce96af None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:02.514592 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acecd2-ecee-460c-9a12-812b58ce96af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:02.530796 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-b2acecd2-ecee-460c-9a12-812b58ce96af None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:02.531886 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:02.532029 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-b2acecd2-ecee-460c-9a12-812b58ce96af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:02.535265 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:02.585126 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6d9d8-f58e-4721-bc67-79c9ac3c8d33 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:02.601009 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 397/809] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:02.602315 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2acecd2-ecee-460c-9a12-812b58ce96af None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:02.629612 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ec7281-3b2b-4c95-a49a-011f7d6297fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:02.630228 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec7281-3b2b-4c95-a49a-011f7d6297fc None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:02.630505 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec7281-3b2b-4c95-a49a-011f7d6297fc None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:02.630775 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec7281-3b2b-4c95-a49a-011f7d6297fc None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:02.631537 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ec7281-3b2b-4c95-a49a-011f7d6297fc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:02.631830 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ec7281-3b2b-4c95-a49a-011f7d6297fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:02.633263 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ec7281-3b2b-4c95-a49a-011f7d6297fc None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:02.651279 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 413/810] 10.222.0.33 () {56 vars in 1149 bytes} [Sun Sep 11 14:55:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:02.652082 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ec7281-3b2b-4c95-a49a-011f7d6297fc None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:02.720256 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2acecd2-ecee-460c-9a12-812b58ce96af None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:02.755080 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ec7281-3b2b-4c95-a49a-011f7d6297fc None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:02.757541 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15fa34d-4c4c-41cc-a8d9-1fd032644468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:02.758304 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fa34d-4c4c-41cc-a8d9-1fd032644468 None admin] REQUEST_METHOD: `PUT` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:02.758584 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fa34d-4c4c-41cc-a8d9-1fd032644468 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:02.758971 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fa34d-4c4c-41cc-a8d9-1fd032644468 None admin] PATH_INFO: `/v3/projects/9e4a3d44808e4b4092cfc4ee284c0176/users/698f1293294d45b89e04d85649f454d0/roles/99e8fe7df8e7446680a81d4eaaa1cb68` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:02.760858 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15fa34d-4c4c-41cc-a8d9-1fd032644468 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e4a3d44808e4b4092cfc4ee284c0176, user_id=698f1293294d45b89e04d85649f454d0, role_id=99e8fe7df8e7446680a81d4eaaa1cb68)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:02.762601 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:02.763045 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15fa34d-4c4c-41cc-a8d9-1fd032644468 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:02.773781 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:02.780193 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 398/811] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 14:55:02 2022] PUT /identity/v3/projects/9e4a3d44808e4b4092cfc4ee284c0176/users/698f1293294d45b89e04d85649f454d0/roles/99e8fe7df8e7446680a81d4eaaa1cb68 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:55:02.792112 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1a7e4-c3df-4d2c-966a-d06af53f33e5 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:02.793017 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1a7e4-c3df-4d2c-966a-d06af53f33e5 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:02.793381 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1a7e4-c3df-4d2c-966a-d06af53f33e5 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:02.799772 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 399/812] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:02 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:02.800961 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad1a7e4-c3df-4d2c-966a-d06af53f33e5 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:02.806976 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9beb0f9-9e07-4fed-a057-a1c6ddebbd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:02.807905 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9beb0f9-9e07-4fed-a057-a1c6ddebbd11 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:02.808597 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9beb0f9-9e07-4fed-a057-a1c6ddebbd11 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:02.809070 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9beb0f9-9e07-4fed-a057-a1c6ddebbd11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:02.898633 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9beb0f9-9e07-4fed-a057-a1c6ddebbd11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:02.900160 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9beb0f9-9e07-4fed-a057-a1c6ddebbd11 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:02.925321 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad1a7e4-c3df-4d2c-966a-d06af53f33e5 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:02.964019 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff78a7-2cdf-4539-a14e-4997022d5f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:02.964819 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff78a7-2cdf-4539-a14e-4997022d5f51 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:02.965209 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff78a7-2cdf-4539-a14e-4997022d5f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:02.965489 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff78a7-2cdf-4539-a14e-4997022d5f51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:02.977677 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ff78a7-2cdf-4539-a14e-4997022d5f51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:02.979919 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ff78a7-2cdf-4539-a14e-4997022d5f51 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:02.997190 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-81ff78a7-2cdf-4539-a14e-4997022d5f51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:02.997190 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:02.997190 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:02.997190 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:02.997190 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:02.997190 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:02.997190 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:02.997190 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:02.997190 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:02.997190 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:02.997190 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:02.997190 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:02.997190 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:55:02.997190 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:55:02.997190 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:55:02.997190 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:55:02.998251 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:02.998251 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:02.998251 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:02.998251 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:02.998251 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:02.998251 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:02.998251 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:02.998251 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:02.998251 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:02.998251 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:02.998251 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:02.998251 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:02.998251 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:02.998251 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:02.998251 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:02.998251 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:02.998251 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:55:02.999336 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:55:02.999336 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:02.999336 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:02.999336 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 400/813] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:55:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:02.999872 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c4f05-6961-4836-b183-15ba5ffb0a3c None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:03.000154 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c4f05-6961-4836-b183-15ba5ffb0a3c None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:03.000428 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c4f05-6961-4836-b183-15ba5ffb0a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:03.020174 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-fb6c4f05-6961-4836-b183-15ba5ffb0a3c None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:03.021737 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-fb6c4f05-6961-4836-b183-15ba5ffb0a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:03.037347 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 414/814] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:55:02 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 14:55:03.038036 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9beb0f9-9e07-4fed-a057-a1c6ddebbd11 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:03.046481 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:03.076199 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:03.101234 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 401/815] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:03.111702 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c4f05-6961-4836-b183-15ba5ffb0a3c None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:03.148450 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9beb0f9-9e07-4fed-a057-a1c6ddebbd11 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:03.194873 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e934e-c9b5-4fbb-8f35-bc9612f96051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:03.195441 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e934e-c9b5-4fbb-8f35-bc9612f96051 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:03.195727 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e934e-c9b5-4fbb-8f35-bc9612f96051 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:03.195999 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e934e-c9b5-4fbb-8f35-bc9612f96051 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:03.196589 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e934e-c9b5-4fbb-8f35-bc9612f96051 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:03.197970 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e934e-c9b5-4fbb-8f35-bc9612f96051 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:03.209711 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-2f8e934e-c9b5-4fbb-8f35-bc9612f96051 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 14:55:03.209711 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:03.209711 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:55:03.209711 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:55:03.209711 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:55:03.209711 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:55:03.209711 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:55:03.209711 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:03.209711 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 14:55:03.209711 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:03.209711 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:55:03.209711 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:03.209711 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:03.209711 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:03.209711 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:03.209711 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:03.209711 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:03.214991 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:03.214991 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:03.214991 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:03.214991 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:03.214991 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:03.214991 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:03.214991 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:55:03.214991 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:55:03.214991 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:55:03.214991 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:55:03.214991 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:03.214991 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:03.214991 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:03.214991 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:03.214991 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:03.214991 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:03.215932 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:03.215932 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:03.215932 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:03.215932 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:03.215932 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:03.215932 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:03.215932 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:03.215932 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:03.215932 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:03.215932 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:03.215932 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:55:03.215932 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:03.215932 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 14:55:03.215932 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:03.215932 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 415/816] 10.222.0.33 () {56 vars in 1129 bytes} [Sun Sep 11 14:55:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:03.216955 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e934e-c9b5-4fbb-8f35-bc9612f96051 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:03.257147 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c4f05-6961-4836-b183-15ba5ffb0a3c None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:03.294762 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d812e3a-224f-4686-9f56-95969938ab47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:03.295888 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d812e3a-224f-4686-9f56-95969938ab47 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:03.296365 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d812e3a-224f-4686-9f56-95969938ab47 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:03.296778 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d812e3a-224f-4686-9f56-95969938ab47 None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:03.297574 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d812e3a-224f-4686-9f56-95969938ab47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:03.298026 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d812e3a-224f-4686-9f56-95969938ab47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:03.299586 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d812e3a-224f-4686-9f56-95969938ab47 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:03.299722 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:03.318508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 402/817] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:03.319648 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d812e3a-224f-4686-9f56-95969938ab47 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:03.341118 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e934e-c9b5-4fbb-8f35-bc9612f96051 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:03.368083 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:03.385364 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de2fd9-4b48-430b-b625-2872cf47a6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:03.386000 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de2fd9-4b48-430b-b625-2872cf47a6c0 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:03.386281 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de2fd9-4b48-430b-b625-2872cf47a6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:03.386553 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de2fd9-4b48-430b-b625-2872cf47a6c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:03.398003 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3de2fd9-4b48-430b-b625-2872cf47a6c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:03.399632 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3de2fd9-4b48-430b-b625-2872cf47a6c0 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:03.415201 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-c3de2fd9-4b48-430b-b625-2872cf47a6c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:03.415201 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:03.415201 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:03.415201 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:03.415201 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:03.415201 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:03.415201 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:03.415201 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:03.415201 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:03.415201 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:03.415201 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:03.415201 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:03.415201 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:55:03.415201 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:55:03.415201 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:55:03.415201 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:55:03.416166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:03.416166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:03.416166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:03.416166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:03.416166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:03.416166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:03.416166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:03.416166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:03.416166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:03.416166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:03.416166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:03.416166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:03.416166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:03.416166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:03.416166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:03.416166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:03.416166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:55:03.417086 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:55:03.417086 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:03.417086 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:03.417086 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 416/818] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:55:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:03.417086 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de2fd9-4b48-430b-b625-2872cf47a6c0 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:03.434888 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d812e3a-224f-4686-9f56-95969938ab47 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:03.472526 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf0bd75-3983-40db-87aa-3e2e37ad24df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:03.473415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0bd75-3983-40db-87aa-3e2e37ad24df None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:03.473800 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0bd75-3983-40db-87aa-3e2e37ad24df None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:03.474176 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0bd75-3983-40db-87aa-3e2e37ad24df None admin] PATH_INFO: `/v3/services` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:03.474939 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf0bd75-3983-40db-87aa-3e2e37ad24df None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:03.475343 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf0bd75-3983-40db-87aa-3e2e37ad24df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:03.477076 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf0bd75-3983-40db-87aa-3e2e37ad24df None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:03.487328 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 403/819] 10.222.0.33 () {56 vars in 1124 bytes} [Sun Sep 11 14:55:03 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:03.487328 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf0bd75-3983-40db-87aa-3e2e37ad24df None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:03.527773 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de2fd9-4b48-430b-b625-2872cf47a6c0 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:03.552123 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:03.565645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d499f-fc43-4cbc-844c-5865e1a9dcf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:03.566366 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d499f-fc43-4cbc-844c-5865e1a9dcf7 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:03.566709 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d499f-fc43-4cbc-844c-5865e1a9dcf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:03.567584 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d499f-fc43-4cbc-844c-5865e1a9dcf7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:03.574010 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847d499f-fc43-4cbc-844c-5865e1a9dcf7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:03.575581 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847d499f-fc43-4cbc-844c-5865e1a9dcf7 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:03.587763 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-847d499f-fc43-4cbc-844c-5865e1a9dcf7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:55:03.587763 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:03.587763 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:03.587763 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:03.587763 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:03.587763 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:03.587763 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:03.587763 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:03.587763 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:03.587763 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:03.587763 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:03.587763 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:03.587763 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:55:03.587763 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:55:03.587763 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:55:03.588860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:55:03.588860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:03.588860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:03.588860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:03.588860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:03.588860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:03.588860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:03.588860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:03.588860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:03.588860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:03.588860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:03.588860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:03.588860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:03.588860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:03.588860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:03.588860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:03.588860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:03.589958 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:55:03.589958 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:55:03.589958 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:55:03.589958 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:55:03.589958 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:55:03.589958 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:55:03.589958 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:55:03.589958 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:03.590636 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 417/820] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:55:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:03.591432 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d499f-fc43-4cbc-844c-5865e1a9dcf7 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:03.610038 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf0bd75-3983-40db-87aa-3e2e37ad24df None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:03.620469 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:03.652591 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f777d-5cdd-45c7-8ddf-c140f99d1ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:03.653639 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f777d-5cdd-45c7-8ddf-c140f99d1ac1 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:03.654480 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f777d-5cdd-45c7-8ddf-c140f99d1ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:03.655083 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f777d-5cdd-45c7-8ddf-c140f99d1ac1 None admin] PATH_INFO: `/v3/services/network` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:03.655947 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1f777d-5cdd-45c7-8ddf-c140f99d1ac1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:03.657469 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1f777d-5cdd-45c7-8ddf-c140f99d1ac1 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:03.667663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-da1f777d-5cdd-45c7-8ddf-c140f99d1ac1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 14:55:03.667663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:03.667663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:55:03.667663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:55:03.667663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:55:03.667663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:55:03.667663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:55:03.667663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:03.667663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 14:55:03.667663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:03.667663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:55:03.667663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:03.667663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:03.667663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:03.667663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:03.667663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:03.667663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:03.668874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:03.668874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:03.668874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:03.668874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:03.668874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:03.668874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:03.668874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:55:03.668874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:55:03.668874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:55:03.668874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:55:03.668874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:03.668874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:03.668874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:03.668874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:03.668874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:03.668874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:03.669938 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:03.669938 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:03.669938 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:03.669938 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:03.669938 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:03.669938 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:03.669938 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:03.669938 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:03.669938 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:03.669938 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:03.669938 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:55:03.669938 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:03.669938 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 14:55:03.669938 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:03.672977 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 404/821] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:55:03 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:03.673922 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f777d-5cdd-45c7-8ddf-c140f99d1ac1 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:03.703666 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d499f-fc43-4cbc-844c-5865e1a9dcf7 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:03.739285 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6289735-0a2a-4994-94b6-5445b423ef3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:03.739891 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6289735-0a2a-4994-94b6-5445b423ef3c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:03.740173 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6289735-0a2a-4994-94b6-5445b423ef3c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:03.740446 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6289735-0a2a-4994-94b6-5445b423ef3c None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:03.741079 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6289735-0a2a-4994-94b6-5445b423ef3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:03.741375 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6289735-0a2a-4994-94b6-5445b423ef3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:03.742770 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6289735-0a2a-4994-94b6-5445b423ef3c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:03.754839 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 418/822] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:03.755660 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6289735-0a2a-4994-94b6-5445b423ef3c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:03.797407 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f777d-5cdd-45c7-8ddf-c140f99d1ac1 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:03.799977 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:03.833132 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4689cfdd-144d-4ee2-89b2-563825b5b8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:03.833741 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689cfdd-144d-4ee2-89b2-563825b5b8eb None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:03.834021 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689cfdd-144d-4ee2-89b2-563825b5b8eb None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:03.834297 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689cfdd-144d-4ee2-89b2-563825b5b8eb None admin] PATH_INFO: `/v3/services` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:03.834955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4689cfdd-144d-4ee2-89b2-563825b5b8eb None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:03.835264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4689cfdd-144d-4ee2-89b2-563825b5b8eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:03.836634 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4689cfdd-144d-4ee2-89b2-563825b5b8eb None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:03.841711 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 405/823] 10.222.0.33 () {56 vars in 1124 bytes} [Sun Sep 11 14:55:03 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:03.843987 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4689cfdd-144d-4ee2-89b2-563825b5b8eb None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:03.867423 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6289735-0a2a-4994-94b6-5445b423ef3c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:03.880381 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:03.907821 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c3d5b-c1ca-489f-a9ba-ec18f85130d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:03.908755 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c3d5b-c1ca-489f-a9ba-ec18f85130d4 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:03.909043 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c3d5b-c1ca-489f-a9ba-ec18f85130d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:03.909467 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c3d5b-c1ca-489f-a9ba-ec18f85130d4 None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:03.910156 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689c3d5b-c1ca-489f-a9ba-ec18f85130d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:03.910552 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689c3d5b-c1ca-489f-a9ba-ec18f85130d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:03.911929 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689c3d5b-c1ca-489f-a9ba-ec18f85130d4 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:03.928660 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 419/824] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:55:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:03.928660 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c3d5b-c1ca-489f-a9ba-ec18f85130d4 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:03.958532 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4689cfdd-144d-4ee2-89b2-563825b5b8eb None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:04.000254 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac760d4-3fb9-4fb1-8fd3-e088a12a8051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:04.001035 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac760d4-3fb9-4fb1-8fd3-e088a12a8051 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:04.001323 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac760d4-3fb9-4fb1-8fd3-e088a12a8051 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:04.001910 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac760d4-3fb9-4fb1-8fd3-e088a12a8051 None admin] PATH_INFO: `/v3/services` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:04.002557 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac760d4-3fb9-4fb1-8fd3-e088a12a8051 None admin] RBAC: Adding query filter params (name=network) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:04.002892 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac760d4-3fb9-4fb1-8fd3-e088a12a8051 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:04.004268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac760d4-3fb9-4fb1-8fd3-e088a12a8051 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:04.009559 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 406/825] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 11 14:55:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:04.010166 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac760d4-3fb9-4fb1-8fd3-e088a12a8051 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:04.047690 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c3d5b-c1ca-489f-a9ba-ec18f85130d4 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:04.050137 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:04.086622 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997c65b-c9a2-49c6-9e11-ff21c5bceece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:04.088035 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997c65b-c9a2-49c6-9e11-ff21c5bceece None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:04.088035 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997c65b-c9a2-49c6-9e11-ff21c5bceece None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:04.088035 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997c65b-c9a2-49c6-9e11-ff21c5bceece None admin] PATH_INFO: `/v3/projects/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:04.095749 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6997c65b-c9a2-49c6-9e11-ff21c5bceece None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:04.097336 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6997c65b-c9a2-49c6-9e11-ff21c5bceece None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:04.110450 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-6997c65b-c9a2-49c6-9e11-ff21c5bceece None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:55:04.110450 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:04.110450 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:04.110450 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:04.110450 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:04.110450 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:04.110450 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:04.110450 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:04.110450 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:04.110450 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:04.110450 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:04.110450 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:04.110450 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:55:04.110450 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:55:04.110450 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:55:04.111607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:55:04.111607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:04.111607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:04.111607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:04.111607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:04.111607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:04.111607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:04.111607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:04.111607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:04.111607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:04.111607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:04.111607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:04.111607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:04.111607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:04.111607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:04.111607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:04.111607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:04.112546 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:55:04.112546 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:55:04.112546 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:55:04.112546 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:55:04.112546 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:55:04.112546 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:55:04.112546 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:55:04.112546 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:04.112546 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 420/826] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:55:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:04.113025 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997c65b-c9a2-49c6-9e11-ff21c5bceece None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:04.119639 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:04.124573 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac760d4-3fb9-4fb1-8fd3-e088a12a8051 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:04.161877 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d91d6c3-02f8-4e4f-823d-69c7b0e64f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:04.163091 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91d6c3-02f8-4e4f-823d-69c7b0e64f36 None admin] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:04.163519 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91d6c3-02f8-4e4f-823d-69c7b0e64f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:04.163799 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91d6c3-02f8-4e4f-823d-69c7b0e64f36 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:04.164476 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d91d6c3-02f8-4e4f-823d-69c7b0e64f36 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:04.165854 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d91d6c3-02f8-4e4f-823d-69c7b0e64f36 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:04.190193 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 407/827] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 14:55:04 2022] POST /identity/v3/endpoints => generated 346 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:55:04.191405 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d91d6c3-02f8-4e4f-823d-69c7b0e64f36 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:04.226792 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997c65b-c9a2-49c6-9e11-ff21c5bceece None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:04.270980 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000c82e-0de2-40ae-9126-46d9e4dddeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:04.271789 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000c82e-0de2-40ae-9126-46d9e4dddeea None admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:04.272077 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000c82e-0de2-40ae-9126-46d9e4dddeea None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:04.272357 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000c82e-0de2-40ae-9126-46d9e4dddeea None admin] PATH_INFO: `/v3/projects/9e4a3d44808e4b4092cfc4ee284c0176/users/0aafb2bbba1d466fb22bb0824a531ddc/roles/533fa341c4f64cd4b0a7ebb668f9106a` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:04.289303 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5000c82e-0de2-40ae-9126-46d9e4dddeea None admin] RBAC: Authorizing `identity:create_grant(project_id=9e4a3d44808e4b4092cfc4ee284c0176, user_id=0aafb2bbba1d466fb22bb0824a531ddc, role_id=533fa341c4f64cd4b0a7ebb668f9106a)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:04.290783 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5000c82e-0de2-40ae-9126-46d9e4dddeea None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:04.308429 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 421/828] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 14:55:04 2022] PUT /identity/v3/projects/9e4a3d44808e4b4092cfc4ee284c0176/users/0aafb2bbba1d466fb22bb0824a531ddc/roles/533fa341c4f64cd4b0a7ebb668f9106a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:55:04.309121 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000c82e-0de2-40ae-9126-46d9e4dddeea None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:04.321807 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:04.352720 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d91d6c3-02f8-4e4f-823d-69c7b0e64f36 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:04.439370 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be69e2-f742-498c-97c8-4ec2eee8fdf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:04.440013 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be69e2-f742-498c-97c8-4ec2eee8fdf1 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:04.440544 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be69e2-f742-498c-97c8-4ec2eee8fdf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:04.440830 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be69e2-f742-498c-97c8-4ec2eee8fdf1 None admin] PATH_INFO: `/v3/services` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:04.441683 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be69e2-f742-498c-97c8-4ec2eee8fdf1 None admin] RBAC: Adding query filter params (type=network) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:04.441994 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be69e2-f742-498c-97c8-4ec2eee8fdf1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:04.443465 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be69e2-f742-498c-97c8-4ec2eee8fdf1 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:04.449596 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 408/829] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 11 14:55:04 2022] GET /identity/v3/services?type=network => generated 350 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:04.457716 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be69e2-f742-498c-97c8-4ec2eee8fdf1 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:04.486501 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:04.531546 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000c82e-0de2-40ae-9126-46d9e4dddeea None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:04.579926 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1a1c1-8f1f-4c9c-accc-dea5a5f12870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:04.580486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1a1c1-8f1f-4c9c-accc-dea5a5f12870 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:04.580768 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1a1c1-8f1f-4c9c-accc-dea5a5f12870 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:04.581042 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1a1c1-8f1f-4c9c-accc-dea5a5f12870 None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:04.581762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1a1c1-8f1f-4c9c-accc-dea5a5f12870 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:04.582070 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1a1c1-8f1f-4c9c-accc-dea5a5f12870 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:04.583600 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1a1c1-8f1f-4c9c-accc-dea5a5f12870 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:04.601740 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 422/830] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:55:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:04.608250 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1a1c1-8f1f-4c9c-accc-dea5a5f12870 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:04.690205 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:04.750630 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be69e2-f742-498c-97c8-4ec2eee8fdf1 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:04.758741 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1a1c1-8f1f-4c9c-accc-dea5a5f12870 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:04.822002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015616b5-d4e3-46f8-baa5-5d0ac4a89e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:04.823704 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015616b5-d4e3-46f8-baa5-5d0ac4a89e0d None admin] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:04.824230 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015616b5-d4e3-46f8-baa5-5d0ac4a89e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:04.825390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015616b5-d4e3-46f8-baa5-5d0ac4a89e0d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:04.827300 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015616b5-d4e3-46f8-baa5-5d0ac4a89e0d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:04.829403 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015616b5-d4e3-46f8-baa5-5d0ac4a89e0d None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:04.834646 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59466119-9076-43b3-8d40-321093348edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:04.835560 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59466119-9076-43b3-8d40-321093348edd None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:04.836054 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59466119-9076-43b3-8d40-321093348edd None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:04.836522 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59466119-9076-43b3-8d40-321093348edd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:04.837448 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59466119-9076-43b3-8d40-321093348edd None admin] RBAC: Adding query filter params (role.id=99e8fe7df8e7446680a81d4eaaa1cb68, scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176, user.id=fc0ea93084b0482fb2dd6d802987942f) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:04.837972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59466119-9076-43b3-8d40-321093348edd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:04.839623 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59466119-9076-43b3-8d40-321093348edd None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:04.858837 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 423/831] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:55:04 2022] GET /identity/v3/role_assignments?user.id=fc0ea93084b0482fb2dd6d802987942f&scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176&role.id=99e8fe7df8e7446680a81d4eaaa1cb68 => generated 608 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:04.885769 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 409/832] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 14:55:04 2022] POST /identity/v3/endpoints => generated 337 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:55:05.049397 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6484d7-8db5-4275-be9e-c78698f8cf77 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:05.049755 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6484d7-8db5-4275-be9e-c78698f8cf77 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:05.049934 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6484d7-8db5-4275-be9e-c78698f8cf77 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:05.051660 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 424/833] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:05.057912 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4529ac8f-80d7-40b5-8342-811820dda151 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:05.058517 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4529ac8f-80d7-40b5-8342-811820dda151 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:05.058781 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4529ac8f-80d7-40b5-8342-811820dda151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:05.087679 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-4529ac8f-80d7-40b5-8342-811820dda151 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:05.088844 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-4529ac8f-80d7-40b5-8342-811820dda151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:05.176909 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:05.187019 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 410/834] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:05 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:05.469754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6484d7-8db5-4275-be9e-c78698f8cf77 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:05.574561 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6484d7-8db5-4275-be9e-c78698f8cf77 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:05.604649 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:05.610326 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9d335-3bb8-4e15-8885-af0e0d778f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:05.611367 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9d335-3bb8-4e15-8885-af0e0d778f43 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:05.611714 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9d335-3bb8-4e15-8885-af0e0d778f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:05.612051 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9d335-3bb8-4e15-8885-af0e0d778f43 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:05.623644 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad9d335-3bb8-4e15-8885-af0e0d778f43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:55:05.628634 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-0ad9d335-3bb8-4e15-8885-af0e0d778f43 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:55:05.628634 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:05.628634 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:05.628634 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:05.628634 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:05.628634 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:05.628634 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:05.628634 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:05.628634 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:05.628634 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:05.628634 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:05.628634 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:05.628634 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:55:05.628634 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:55:05.628634 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:55:05.628634 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:55:05.629762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:55:05.629762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:55:05.629762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:55:05.629762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:55:05.629762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:05.629762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:05.629762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:05.629762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:05.629762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:05.629762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:05.629762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:05.629762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:05.629762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:05.629762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:05.629762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:05.629762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:05.630829 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:05.630829 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:05.630829 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:05.630829 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:05.630829 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:55:05.630829 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:55:05.630829 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:55:05.630829 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:55:05.630829 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:55:05.630829 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:55:05.630829 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:55:05.630829 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:05.636795 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 425/835] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:55:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:05.638259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4529ac8f-80d7-40b5-8342-811820dda151 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:05.742144 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4529ac8f-80d7-40b5-8342-811820dda151 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:05.778008 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5221bfad-eed4-4a3e-a858-7a11df6b1a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:05.778739 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221bfad-eed4-4a3e-a858-7a11df6b1a37 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:05.779111 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221bfad-eed4-4a3e-a858-7a11df6b1a37 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:05.779718 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221bfad-eed4-4a3e-a858-7a11df6b1a37 None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:05.780451 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5221bfad-eed4-4a3e-a858-7a11df6b1a37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:05.780819 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5221bfad-eed4-4a3e-a858-7a11df6b1a37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:05.782248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5221bfad-eed4-4a3e-a858-7a11df6b1a37 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:05.795430 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 411/836] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:55:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:05.797911 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9d335-3bb8-4e15-8885-af0e0d778f43 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:05.905485 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9d335-3bb8-4e15-8885-af0e0d778f43 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:05.940707 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc6191-0442-4b4f-a8e8-a2c73ecc1d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:05.941421 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc6191-0442-4b4f-a8e8-a2c73ecc1d43 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:05.941763 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc6191-0442-4b4f-a8e8-a2c73ecc1d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:05.942102 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc6191-0442-4b4f-a8e8-a2c73ecc1d43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:05.953442 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc6191-0442-4b4f-a8e8-a2c73ecc1d43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:05.954988 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc6191-0442-4b4f-a8e8-a2c73ecc1d43 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:05.969537 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-2cfc6191-0442-4b4f-a8e8-a2c73ecc1d43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:05.969537 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:05.969537 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:05.969537 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:05.969537 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:05.969537 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:05.969537 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:05.969537 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:05.969537 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:05.969537 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:05.969537 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:05.969537 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:05.969537 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:55:05.969537 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:55:05.969537 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:55:05.969537 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:55:05.970673 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:05.970673 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:05.970673 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:05.970673 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:05.970673 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:05.970673 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:05.970673 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:05.970673 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:05.970673 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:05.970673 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:05.970673 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:05.970673 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:05.970673 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:05.970673 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:05.970673 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:05.970673 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:05.970673 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:55:05.971851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:55:05.971851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:05.971851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:05.976921 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 426/837] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:55:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:05.978304 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5221bfad-eed4-4a3e-a858-7a11df6b1a37 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:06.044324 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:06.087263 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5221bfad-eed4-4a3e-a858-7a11df6b1a37 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:06.124499 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d845e126-1f6a-417f-8cab-662ebf27d3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:06.125107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845e126-1f6a-417f-8cab-662ebf27d3c9 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:06.125436 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845e126-1f6a-417f-8cab-662ebf27d3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:06.125707 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845e126-1f6a-417f-8cab-662ebf27d3c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:06.126342 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d845e126-1f6a-417f-8cab-662ebf27d3c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:06.126642 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d845e126-1f6a-417f-8cab-662ebf27d3c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:06.128039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d845e126-1f6a-417f-8cab-662ebf27d3c9 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:06.144431 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 412/838] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:55:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:06.146006 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc6191-0442-4b4f-a8e8-a2c73ecc1d43 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:06.198294 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:06.255911 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc6191-0442-4b4f-a8e8-a2c73ecc1d43 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:06.291536 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ff5a8-7d38-4b55-80f2-e78658a9d946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:06.292170 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ff5a8-7d38-4b55-80f2-e78658a9d946 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:06.292454 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ff5a8-7d38-4b55-80f2-e78658a9d946 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:06.292725 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ff5a8-7d38-4b55-80f2-e78658a9d946 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:06.300767 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351ff5a8-7d38-4b55-80f2-e78658a9d946 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:06.302227 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351ff5a8-7d38-4b55-80f2-e78658a9d946 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:06.315323 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-351ff5a8-7d38-4b55-80f2-e78658a9d946 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 14:55:06.315323 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:06.315323 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:06.315323 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:06.315323 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:06.315323 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:06.315323 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:06.315323 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:06.315323 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:06.315323 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:06.315323 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:06.315323 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:06.315323 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:55:06.315323 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:55:06.315323 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:55:06.315323 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:55:06.316324 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:06.316324 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:06.316324 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:55:06.316324 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:55:06.316324 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:55:06.316324 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:55:06.316324 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:06.316324 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:06.316324 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:06.316324 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:06.316324 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:06.316324 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:06.316324 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:06.316324 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:06.316324 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:06.316324 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:06.316324 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:06.316324 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:06.317284 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:06.317284 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:06.317284 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:55:06.317284 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:55:06.317284 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:55:06.317284 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:55:06.317284 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:55:06.317284 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:55:06.317284 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 14:55:06.317284 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:06.317284 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 427/839] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:55:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:06.322893 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d845e126-1f6a-417f-8cab-662ebf27d3c9 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:06.429379 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d845e126-1f6a-417f-8cab-662ebf27d3c9 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:06.454696 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:06.468349 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a60f4-3cc5-464a-8cca-b2a54b814164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:06.468953 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a60f4-3cc5-464a-8cca-b2a54b814164 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:06.469236 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a60f4-3cc5-464a-8cca-b2a54b814164 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:06.469485 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a60f4-3cc5-464a-8cca-b2a54b814164 None admin] PATH_INFO: `/v3/users` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:06.470207 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9a60f4-3cc5-464a-8cca-b2a54b814164 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:06.470499 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9a60f4-3cc5-464a-8cca-b2a54b814164 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:06.471996 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9a60f4-3cc5-464a-8cca-b2a54b814164 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:06.492788 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 413/840] 10.222.0.33 () {56 vars in 1135 bytes} [Sun Sep 11 14:55:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:06.499101 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ff5a8-7d38-4b55-80f2-e78658a9d946 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:06.609102 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ff5a8-7d38-4b55-80f2-e78658a9d946 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:06.611697 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:06.645902 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f14cd22-8795-42d8-b364-71c3cf2f2599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:06.646544 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14cd22-8795-42d8-b364-71c3cf2f2599 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:06.646837 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14cd22-8795-42d8-b364-71c3cf2f2599 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:06.647134 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14cd22-8795-42d8-b364-71c3cf2f2599 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:06.727080 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f14cd22-8795-42d8-b364-71c3cf2f2599 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:06.728567 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f14cd22-8795-42d8-b364-71c3cf2f2599 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:06.780933 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69b49d-57b8-4916-acc0-096c30fb27d9 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:06.781462 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69b49d-57b8-4916-acc0-096c30fb27d9 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:06.781462 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69b49d-57b8-4916-acc0-096c30fb27d9 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:06.782796 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 414/841] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:06.792128 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948da6a-79c1-4b00-9ebe-8fe03638d6cc None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:06.792486 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948da6a-79c1-4b00-9ebe-8fe03638d6cc None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:06.792843 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948da6a-79c1-4b00-9ebe-8fe03638d6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:06.813562 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-5948da6a-79c1-4b00-9ebe-8fe03638d6cc None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:06.815124 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-5948da6a-79c1-4b00-9ebe-8fe03638d6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:06.844515 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 428/842] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:55:06 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 14:55:06.846376 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795eeeb-cc38-4d83-83eb-c1965feb107c None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:06.846853 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795eeeb-cc38-4d83-83eb-c1965feb107c None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:06.847339 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795eeeb-cc38-4d83-83eb-c1965feb107c None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:06.849018 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 429/843] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:06.850303 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c795eeeb-cc38-4d83-83eb-c1965feb107c None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:06.857026 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:06.894581 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 415/844] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:06 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:06.896454 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c8467-6c86-4d09-909f-2d8b75683eb3 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:06.896735 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c8467-6c86-4d09-909f-2d8b75683eb3 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:06.897008 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c8467-6c86-4d09-909f-2d8b75683eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:06.923043 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-292c8467-6c86-4d09-909f-2d8b75683eb3 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:06.924528 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-292c8467-6c86-4d09-909f-2d8b75683eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:06.995388 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c795eeeb-cc38-4d83-83eb-c1965feb107c None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:07.031513 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 416/845] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:06 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:07.042040 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b2c06-805e-4349-a268-05ba722a3d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:07.042977 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b2c06-805e-4349-a268-05ba722a3d5e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:07.043463 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b2c06-805e-4349-a268-05ba722a3d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:07.043866 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b2c06-805e-4349-a268-05ba722a3d5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:07.056476 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47b2c06-805e-4349-a268-05ba722a3d5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:07.058520 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47b2c06-805e-4349-a268-05ba722a3d5e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:07.079003 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-b47b2c06-805e-4349-a268-05ba722a3d5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:07.079003 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:07.079003 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:07.079003 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:07.079003 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:07.079003 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:07.079003 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:07.079003 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:07.079003 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:07.079003 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:07.079003 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:07.079003 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:07.079003 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:55:07.079003 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:55:07.079003 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:55:07.079003 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:55:07.082150 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:07.082150 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:07.082150 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:07.082150 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:07.082150 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:07.082150 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:07.082150 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:07.082150 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:07.082150 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:07.082150 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:07.082150 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:07.082150 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:07.082150 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:07.082150 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:07.082150 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:07.082150 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:07.082150 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:55:07.083292 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:55:07.083292 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:07.083292 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:07.085092 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 430/846] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:55:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:07.091219 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c8467-6c86-4d09-909f-2d8b75683eb3 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:07.142769 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:07.212786 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c8467-6c86-4d09-909f-2d8b75683eb3 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:07.249645 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a57376-67fa-4254-915f-974ee45d7f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:07.250518 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a57376-67fa-4254-915f-974ee45d7f29 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:07.250960 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a57376-67fa-4254-915f-974ee45d7f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:07.251379 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a57376-67fa-4254-915f-974ee45d7f29 None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:07.252198 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a57376-67fa-4254-915f-974ee45d7f29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:07.252644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a57376-67fa-4254-915f-974ee45d7f29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:07.254169 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a57376-67fa-4254-915f-974ee45d7f29 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:07.268505 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 417/847] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:55:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:55:07.280027 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b2c06-805e-4349-a268-05ba722a3d5e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:07.293822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a57376-67fa-4254-915f-974ee45d7f29 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:07.362670 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:07.420588 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b2c06-805e-4349-a268-05ba722a3d5e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:07.438612 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:07.442257 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a57376-67fa-4254-915f-974ee45d7f29 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:07.461548 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7268bc08-4148-4016-a9e1-09398a8abc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:07.462187 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268bc08-4148-4016-a9e1-09398a8abc49 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:07.463487 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268bc08-4148-4016-a9e1-09398a8abc49 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:07.463487 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268bc08-4148-4016-a9e1-09398a8abc49 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:07.470674 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7268bc08-4148-4016-a9e1-09398a8abc49 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:07.471248 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7268bc08-4148-4016-a9e1-09398a8abc49 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:07.484519 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-7268bc08-4148-4016-a9e1-09398a8abc49 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:55:07.484519 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:07.484519 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:07.484519 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:07.484519 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:07.484519 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:07.484519 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:07.484519 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:07.484519 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:07.484519 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:07.484519 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:07.484519 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:07.484519 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:55:07.484519 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:55:07.484519 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:55:07.485464 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:55:07.485464 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:07.485464 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:07.485464 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:07.485464 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:07.485464 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:07.485464 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:07.485464 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:07.485464 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:07.485464 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:07.485464 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:07.485464 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:07.485464 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:07.485464 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:07.485464 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:07.485464 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:07.485464 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:07.486353 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:55:07.486353 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:55:07.486353 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:55:07.486353 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:55:07.486353 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:55:07.486353 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:55:07.486353 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:55:07.486353 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:07.486783 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c816f34d-4fdd-4b73-97b9-387f194a8956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:07.486783 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816f34d-4fdd-4b73-97b9-387f194a8956 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:07.487289 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 431/848] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:55:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:07.487360 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816f34d-4fdd-4b73-97b9-387f194a8956 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:07.487360 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816f34d-4fdd-4b73-97b9-387f194a8956 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:07.487578 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7268bc08-4148-4016-a9e1-09398a8abc49 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:07.499958 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c816f34d-4fdd-4b73-97b9-387f194a8956 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:55:07.505032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-c816f34d-4fdd-4b73-97b9-387f194a8956 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:55:07.505032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:07.505032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:07.505032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:07.505032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:07.505032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:07.505032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:07.505032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:07.505032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:07.505032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:07.505032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:07.505032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:07.505032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:55:07.505032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:55:07.505032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:55:07.505032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:55:07.506003 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:55:07.506003 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:55:07.506003 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:55:07.506003 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:55:07.506003 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:07.506003 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:07.506003 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:07.506003 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:07.506003 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:07.506003 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:07.506003 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:07.506003 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:07.506003 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:07.506003 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:07.506003 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:07.506003 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:07.506911 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:07.506911 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:07.506911 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:07.506911 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:07.506911 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:55:07.506911 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:55:07.506911 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:55:07.506911 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:55:07.506911 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:55:07.506911 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:55:07.506911 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:55:07.506911 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:07.506911 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 418/849] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 14:55:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:07.508089 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc877cc-2a30-4367-88c3-cddd42aa8a4f None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:07.508436 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc877cc-2a30-4367-88c3-cddd42aa8a4f None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:07.508774 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc877cc-2a30-4367-88c3-cddd42aa8a4f None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:07.510079 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 419/850] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:07.511446 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc877cc-2a30-4367-88c3-cddd42aa8a4f None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:07.599596 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7268bc08-4148-4016-a9e1-09398a8abc49 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:07.616818 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:07.622827 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc877cc-2a30-4367-88c3-cddd42aa8a4f None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:07.636314 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed95b50-dff0-488a-b8b5-4669dc7d5721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:07.636954 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed95b50-dff0-488a-b8b5-4669dc7d5721 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:07.637239 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed95b50-dff0-488a-b8b5-4669dc7d5721 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:07.637513 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed95b50-dff0-488a-b8b5-4669dc7d5721 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:07.638104 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed95b50-dff0-488a-b8b5-4669dc7d5721 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:07.639552 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed95b50-dff0-488a-b8b5-4669dc7d5721 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:07.648623 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-8ed95b50-dff0-488a-b8b5-4669dc7d5721 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 14:55:07.648623 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:07.648623 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:55:07.648623 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:55:07.648623 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:55:07.648623 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:55:07.648623 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:55:07.648623 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:07.648623 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 14:55:07.648623 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:07.648623 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:55:07.648623 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:07.648623 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:07.648623 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:07.648623 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:07.648623 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:07.648623 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:07.649652 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:07.649652 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:07.649652 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:07.649652 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:07.649652 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:07.649652 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:07.649652 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:55:07.649652 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:55:07.649652 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:55:07.649652 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:55:07.649652 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:07.649652 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:07.649652 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:07.649652 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:07.649652 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:07.649652 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:07.650625 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:07.650625 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:07.650625 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:07.650625 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:07.650625 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:07.650625 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:07.650625 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:07.650625 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:07.650625 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:07.650625 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:07.650625 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:55:07.650625 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:07.650625 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 14:55:07.650625 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:07.650625 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 432/851] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 14:55:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:07.651474 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed95b50-dff0-488a-b8b5-4669dc7d5721 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:07.667788 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda09b1-194c-4482-a8a0-3c49074d5385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:07.668549 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda09b1-194c-4482-a8a0-3c49074d5385 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:07.668828 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda09b1-194c-4482-a8a0-3c49074d5385 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:07.669093 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda09b1-194c-4482-a8a0-3c49074d5385 None admin] PATH_INFO: `/v3/projects` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:07.669783 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda09b1-194c-4482-a8a0-3c49074d5385 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:07.670671 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda09b1-194c-4482-a8a0-3c49074d5385 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:07.672177 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda09b1-194c-4482-a8a0-3c49074d5385 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:07.684668 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 420/852] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:55:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:07.686468 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4e7e8-0552-41a3-93d4-fc71d6bf3222 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:07.687055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4e7e8-0552-41a3-93d4-fc71d6bf3222 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:07.687055 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4e7e8-0552-41a3-93d4-fc71d6bf3222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:07.707032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-74e4e7e8-0552-41a3-93d4-fc71d6bf3222 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:07.708224 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-74e4e7e8-0552-41a3-93d4-fc71d6bf3222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:07.724948 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:07.760732 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed95b50-dff0-488a-b8b5-4669dc7d5721 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:07.786035 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 421/853] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:07 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:07.786962 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4e7e8-0552-41a3-93d4-fc71d6bf3222 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:07.796832 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d828df5-8fe5-462b-99a3-8f095f991a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:07.797525 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d828df5-8fe5-462b-99a3-8f095f991a64 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:07.797872 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d828df5-8fe5-462b-99a3-8f095f991a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:07.798218 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d828df5-8fe5-462b-99a3-8f095f991a64 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:07.798985 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d828df5-8fe5-462b-99a3-8f095f991a64 None admin] RBAC: Adding query filter params (name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:07.799288 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d828df5-8fe5-462b-99a3-8f095f991a64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:07.800663 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d828df5-8fe5-462b-99a3-8f095f991a64 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:07.813638 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 433/854] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 14:55:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:07.813638 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d828df5-8fe5-462b-99a3-8f095f991a64 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:07.860943 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:07.917677 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4e7e8-0552-41a3-93d4-fc71d6bf3222 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:07.920790 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d828df5-8fe5-462b-99a3-8f095f991a64 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:07.955943 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d2da3a-3b0a-4601-be1f-b9c4ddcfdb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:07.956635 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2da3a-3b0a-4601-be1f-b9c4ddcfdb40 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:07.956988 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2da3a-3b0a-4601-be1f-b9c4ddcfdb40 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:07.957325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2da3a-3b0a-4601-be1f-b9c4ddcfdb40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:07.958179 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d2da3a-3b0a-4601-be1f-b9c4ddcfdb40 None admin] RBAC: Adding query filter params (role.id=99e8fe7df8e7446680a81d4eaaa1cb68, scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176, user.id=698f1293294d45b89e04d85649f454d0) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:07.958496 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d2da3a-3b0a-4601-be1f-b9c4ddcfdb40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:07.960074 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d2da3a-3b0a-4601-be1f-b9c4ddcfdb40 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:07.961830 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b481444-f159-40cd-bf9c-34d382192928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:07.962516 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b481444-f159-40cd-bf9c-34d382192928 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:07.963389 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b481444-f159-40cd-bf9c-34d382192928 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:07.963750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b481444-f159-40cd-bf9c-34d382192928 None admin] PATH_INFO: `/v3/services` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:07.964736 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b481444-f159-40cd-bf9c-34d382192928 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:07.965112 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b481444-f159-40cd-bf9c-34d382192928 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:07.975254 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 434/855] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:55:07 2022] GET /identity/v3/role_assignments?user.id=698f1293294d45b89e04d85649f454d0&scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176&role.id=99e8fe7df8e7446680a81d4eaaa1cb68 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:07.975431 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b481444-f159-40cd-bf9c-34d382192928 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:07.975514 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d2da3a-3b0a-4601-be1f-b9c4ddcfdb40 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:07.990439 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 422/856] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 14:55:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:07.991531 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b481444-f159-40cd-bf9c-34d382192928 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:08.031215 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:08.100616 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d2da3a-3b0a-4601-be1f-b9c4ddcfdb40 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:08.110734 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:08.115484 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b481444-f159-40cd-bf9c-34d382192928 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:08.143587 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573c5e1-a856-4eee-b6e9-19699514c8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:08.144231 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573c5e1-a856-4eee-b6e9-19699514c8c5 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:08.144515 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573c5e1-a856-4eee-b6e9-19699514c8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:08.145167 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573c5e1-a856-4eee-b6e9-19699514c8c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:08.157375 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4573c5e1-a856-4eee-b6e9-19699514c8c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:08.158992 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f0feb-a017-405a-83a4-399094942228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:08.159123 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4573c5e1-a856-4eee-b6e9-19699514c8c5 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:08.159374 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f0feb-a017-405a-83a4-399094942228 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:08.159652 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f0feb-a017-405a-83a4-399094942228 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:08.159935 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f0feb-a017-405a-83a4-399094942228 None admin] PATH_INFO: `/v3/services` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:08.160642 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f0feb-a017-405a-83a4-399094942228 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:08.161239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f0feb-a017-405a-83a4-399094942228 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:08.162344 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f0feb-a017-405a-83a4-399094942228 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:08.175307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 423/857] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 14:55:07 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:08.175307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f0feb-a017-405a-83a4-399094942228 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:08.177798 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-4573c5e1-a856-4eee-b6e9-19699514c8c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:08.177798 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:08.177798 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:08.177798 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:08.177798 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:08.177798 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:08.177798 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:08.177798 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:08.177798 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:08.177798 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:08.177798 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:08.177798 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:08.177798 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:55:08.177798 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:55:08.177798 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:55:08.177798 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:55:08.178881 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:08.178881 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:08.178881 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:08.178881 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:08.178881 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:08.178881 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:08.178881 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:08.178881 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:08.178881 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:08.178881 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:08.178881 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:08.178881 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:08.178881 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:08.178881 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:08.178881 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:08.178881 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:08.178881 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:55:08.179788 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:55:08.179788 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:08.179788 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:08.179788 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 435/858] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:55:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:08.184907 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573c5e1-a856-4eee-b6e9-19699514c8c5 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:08.282508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:08.307375 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f0feb-a017-405a-83a4-399094942228 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:08.347154 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5cd106-2498-4291-98c8-a53c8edb90f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:08.347794 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5cd106-2498-4291-98c8-a53c8edb90f6 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:08.348068 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5cd106-2498-4291-98c8-a53c8edb90f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:08.348339 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5cd106-2498-4291-98c8-a53c8edb90f6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:08.348922 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5cd106-2498-4291-98c8-a53c8edb90f6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:08.349622 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573c5e1-a856-4eee-b6e9-19699514c8c5 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:08.350290 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5cd106-2498-4291-98c8-a53c8edb90f6 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:08.360230 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-2a5cd106-2498-4291-98c8-a53c8edb90f6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 14:55:08.360230 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:08.360230 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:55:08.360230 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:55:08.360230 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:55:08.360230 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:55:08.360230 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:55:08.360230 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:08.360230 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 14:55:08.360230 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:08.360230 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:55:08.360230 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:08.360230 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:08.360230 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:08.360230 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:08.360230 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:08.361288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:08.361288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:08.361288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:08.361288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:08.361288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:08.361288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:08.361288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:08.361288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:55:08.361288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:55:08.361288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:55:08.361288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:55:08.361288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:08.361288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:08.361288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:08.361288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:08.361288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:08.361288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:08.362252 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:08.362252 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:08.362252 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:08.362252 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:08.362252 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:08.362252 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:08.362252 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:08.362252 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:08.362252 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:08.362252 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:08.362252 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:55:08.362252 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:08.362252 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 14:55:08.362252 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:08.362252 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 424/859] 10.222.0.33 () {56 vars in 1132 bytes} [Sun Sep 11 14:55:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:08.367481 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5cd106-2498-4291-98c8-a53c8edb90f6 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:08.400685 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e72286-0cfb-4ee3-a6e8-bb90c15e8029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:08.401533 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e72286-0cfb-4ee3-a6e8-bb90c15e8029 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:08.401822 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e72286-0cfb-4ee3-a6e8-bb90c15e8029 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:08.402103 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e72286-0cfb-4ee3-a6e8-bb90c15e8029 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:08.403173 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e72286-0cfb-4ee3-a6e8-bb90c15e8029 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:08.403498 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e72286-0cfb-4ee3-a6e8-bb90c15e8029 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:08.405876 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e72286-0cfb-4ee3-a6e8-bb90c15e8029 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:08.422846 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 436/860] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:55:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:55:08.429260 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e72286-0cfb-4ee3-a6e8-bb90c15e8029 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:08.450829 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:08.491883 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5cd106-2498-4291-98c8-a53c8edb90f6 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:08.534912 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ae4df-8c83-4979-9180-34e6b8ef7573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:08.535505 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ae4df-8c83-4979-9180-34e6b8ef7573 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:08.535795 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ae4df-8c83-4979-9180-34e6b8ef7573 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:08.536072 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ae4df-8c83-4979-9180-34e6b8ef7573 None admin] PATH_INFO: `/v3/services` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:08.536985 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546ae4df-8c83-4979-9180-34e6b8ef7573 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:08.537292 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546ae4df-8c83-4979-9180-34e6b8ef7573 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:08.539494 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546ae4df-8c83-4979-9180-34e6b8ef7573 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:08.549861 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 425/861] 10.222.0.33 () {56 vars in 1126 bytes} [Sun Sep 11 14:55:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:55:08.550053 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e72286-0cfb-4ee3-a6e8-bb90c15e8029 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:08.551676 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ae4df-8c83-4979-9180-34e6b8ef7573 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:08.563735 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:08.596684 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8155551b-11ba-4573-b8c3-862c5e457e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:08.597765 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155551b-11ba-4573-b8c3-862c5e457e46 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:08.598257 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155551b-11ba-4573-b8c3-862c5e457e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:08.598870 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155551b-11ba-4573-b8c3-862c5e457e46 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:08.607377 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8155551b-11ba-4573-b8c3-862c5e457e46 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:08.609710 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8155551b-11ba-4573-b8c3-862c5e457e46 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:08.624844 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-8155551b-11ba-4573-b8c3-862c5e457e46 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 14:55:08.624844 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:08.624844 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:08.624844 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:08.624844 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:08.624844 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:08.624844 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:08.624844 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:08.624844 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:08.624844 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:08.624844 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:08.624844 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:08.624844 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:55:08.624844 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:55:08.624844 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:55:08.624844 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:55:08.625856 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:08.625856 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:08.625856 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:55:08.625856 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:55:08.625856 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:55:08.625856 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:55:08.625856 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:08.625856 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:08.625856 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:08.625856 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:08.625856 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:08.625856 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:08.625856 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:08.625856 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:08.625856 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:08.625856 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:08.625856 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:08.625856 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:08.626897 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:08.626897 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:08.626897 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:55:08.626897 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:55:08.626897 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:55:08.626897 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:55:08.626897 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:55:08.626897 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:55:08.626897 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 14:55:08.626897 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:08.629623 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 437/862] 10.222.0.33 () {56 vars in 1117 bytes} [Sun Sep 11 14:55:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:55:08.637447 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8155551b-11ba-4573-b8c3-862c5e457e46 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:08.663325 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ae4df-8c83-4979-9180-34e6b8ef7573 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:08.698821 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e95e9c-ea80-49a1-a66b-6f5a48d07f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:08.699558 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e95e9c-ea80-49a1-a66b-6f5a48d07f49 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:08.699843 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e95e9c-ea80-49a1-a66b-6f5a48d07f49 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:08.700118 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e95e9c-ea80-49a1-a66b-6f5a48d07f49 None admin] PATH_INFO: `/v3/services` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:08.700748 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e95e9c-ea80-49a1-a66b-6f5a48d07f49 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:08.701054 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e95e9c-ea80-49a1-a66b-6f5a48d07f49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:08.703307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e95e9c-ea80-49a1-a66b-6f5a48d07f49 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:08.708432 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 426/863] 10.222.0.33 () {56 vars in 1126 bytes} [Sun Sep 11 14:55:08 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:55:08.739750 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:08.761726 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8155551b-11ba-4573-b8c3-862c5e457e46 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:08.801499 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb8013-b021-4788-9473-d40af8beba6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:08.802129 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb8013-b021-4788-9473-d40af8beba6c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:08.802409 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb8013-b021-4788-9473-d40af8beba6c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:08.802676 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb8013-b021-4788-9473-d40af8beba6c None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:08.803486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeb8013-b021-4788-9473-d40af8beba6c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:08.803788 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeb8013-b021-4788-9473-d40af8beba6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:08.805320 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeb8013-b021-4788-9473-d40af8beba6c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:08.827307 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 438/864] 10.222.0.33 () {56 vars in 1149 bytes} [Sun Sep 11 14:55:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:08.832608 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e95e9c-ea80-49a1-a66b-6f5a48d07f49 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:08.913163 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:08.947973 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e95e9c-ea80-49a1-a66b-6f5a48d07f49 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:08.984868 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a13d4-2c05-4f76-ac2f-9a7f498f0c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:08.985488 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a13d4-2c05-4f76-ac2f-9a7f498f0c4c None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:08.985799 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a13d4-2c05-4f76-ac2f-9a7f498f0c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:08.986029 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a13d4-2c05-4f76-ac2f-9a7f498f0c4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:09.061756 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49a13d4-2c05-4f76-ac2f-9a7f498f0c4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:09.063162 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49a13d4-2c05-4f76-ac2f-9a7f498f0c4c None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:09.140211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:09.175001 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 427/865] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:55:08 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 14:55:09.180303 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb8013-b021-4788-9473-d40af8beba6c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:09.283175 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb8013-b021-4788-9473-d40af8beba6c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:09.321453 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e78ba-f185-4b35-be5a-a8782c5e35f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:09.322051 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e78ba-f185-4b35-be5a-a8782c5e35f2 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:09.322330 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e78ba-f185-4b35-be5a-a8782c5e35f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:09.322597 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e78ba-f185-4b35-be5a-a8782c5e35f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:09.325332 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:09.333349 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1e78ba-f185-4b35-be5a-a8782c5e35f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:09.335151 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1e78ba-f185-4b35-be5a-a8782c5e35f2 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:09.350791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-0f1e78ba-f185-4b35-be5a-a8782c5e35f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:09.350791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:09.350791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:09.350791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:09.350791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:09.350791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:09.350791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:09.350791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:09.350791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:09.350791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:09.350791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:09.350791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:09.350791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:55:09.350791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:55:09.350791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:55:09.350791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:55:09.351793 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:09.351793 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:09.351793 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:09.351793 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:09.351793 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:09.351793 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:09.351793 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:09.351793 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:09.351793 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:09.351793 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:09.351793 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:09.351793 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:09.351793 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:09.351793 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:09.351793 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:09.351793 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:09.351793 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:55:09.353049 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:55:09.353049 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:09.353049 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:09.353049 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 439/866] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:55:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:09.357604 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a13d4-2c05-4f76-ac2f-9a7f498f0c4c None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:09.469987 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a13d4-2c05-4f76-ac2f-9a7f498f0c4c None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:09.506965 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d2e7df-a4ff-4e9c-9744-dcfa38b26e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:09.507583 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2e7df-a4ff-4e9c-9744-dcfa38b26e0d None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:09.507861 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2e7df-a4ff-4e9c-9744-dcfa38b26e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:09.508137 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2e7df-a4ff-4e9c-9744-dcfa38b26e0d None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:09.508783 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d2e7df-a4ff-4e9c-9744-dcfa38b26e0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:09.509086 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d2e7df-a4ff-4e9c-9744-dcfa38b26e0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:09.510466 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d2e7df-a4ff-4e9c-9744-dcfa38b26e0d None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:09.527157 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 428/867] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:55:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:09.531860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e78ba-f185-4b35-be5a-a8782c5e35f2 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:09.636346 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e78ba-f185-4b35-be5a-a8782c5e35f2 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:09.674079 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141e218-70f0-4454-84fc-be7b097c0030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:09.674702 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141e218-70f0-4454-84fc-be7b097c0030 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:09.675013 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141e218-70f0-4454-84fc-be7b097c0030 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:09.675284 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141e218-70f0-4454-84fc-be7b097c0030 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:09.681847 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4141e218-70f0-4454-84fc-be7b097c0030 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:09.683593 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4141e218-70f0-4454-84fc-be7b097c0030 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:09.695063 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-4141e218-70f0-4454-84fc-be7b097c0030 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:55:09.695063 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:09.695063 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:09.695063 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:09.695063 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:09.695063 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:09.695063 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:09.695063 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:09.695063 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:09.695063 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:09.695063 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:09.695063 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:09.695063 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:55:09.695063 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:55:09.695063 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:55:09.695972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:55:09.695972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:09.695972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:09.695972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:09.695972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:09.695972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:09.695972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:09.695972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:09.695972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:09.695972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:09.695972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:09.695972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:09.695972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:09.695972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:09.695972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:09.695972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:09.695972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:09.696870 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:55:09.696870 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:55:09.696870 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:55:09.696870 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:55:09.696870 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:55:09.696870 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:55:09.696870 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:55:09.696870 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:09.696870 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 440/868] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:55:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:09.704249 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d2e7df-a4ff-4e9c-9744-dcfa38b26e0d None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:09.738363 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:09.816694 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d2e7df-a4ff-4e9c-9744-dcfa38b26e0d None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:09.853485 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816bd1d-ab04-4fcc-aab7-cdc81044af7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:09.854141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816bd1d-ab04-4fcc-aab7-cdc81044af7d None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:09.854374 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816bd1d-ab04-4fcc-aab7-cdc81044af7d None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:09.854626 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816bd1d-ab04-4fcc-aab7-cdc81044af7d None admin] PATH_INFO: `/v3/projects` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:09.855353 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e816bd1d-ab04-4fcc-aab7-cdc81044af7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:09.855658 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e816bd1d-ab04-4fcc-aab7-cdc81044af7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:09.857089 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e816bd1d-ab04-4fcc-aab7-cdc81044af7d None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:09.870477 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 429/869] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:55:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:09.877136 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141e218-70f0-4454-84fc-be7b097c0030 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:09.929751 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:09.985553 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141e218-70f0-4454-84fc-be7b097c0030 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:10.021239 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290528b-bb6e-4c18-8fb9-eac7be89cbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:10.021848 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290528b-bb6e-4c18-8fb9-eac7be89cbb7 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:10.022123 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290528b-bb6e-4c18-8fb9-eac7be89cbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:10.022393 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290528b-bb6e-4c18-8fb9-eac7be89cbb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:10.023140 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a290528b-bb6e-4c18-8fb9-eac7be89cbb7 None admin] RBAC: Adding query filter params (role.id=533fa341c4f64cd4b0a7ebb668f9106a, scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176, user.id=0aafb2bbba1d466fb22bb0824a531ddc) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:10.023447 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a290528b-bb6e-4c18-8fb9-eac7be89cbb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:10.024891 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a290528b-bb6e-4c18-8fb9-eac7be89cbb7 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:10.032379 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 441/870] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:55:09 2022] GET /identity/v3/role_assignments?user.id=0aafb2bbba1d466fb22bb0824a531ddc&scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176&role.id=533fa341c4f64cd4b0a7ebb668f9106a => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:10.386354 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fde8c6-686b-4f93-a9cb-baf0a2f0dc71 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:10.386754 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fde8c6-686b-4f93-a9cb-baf0a2f0dc71 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:10.386978 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fde8c6-686b-4f93-a9cb-baf0a2f0dc71 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:10.389077 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 430/871] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:10.394372 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebec0e-c743-4853-8a57-af0620c5eb77 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:10.394856 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebec0e-c743-4853-8a57-af0620c5eb77 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:10.394959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebec0e-c743-4853-8a57-af0620c5eb77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:10.418227 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-a3ebec0e-c743-4853-8a57-af0620c5eb77 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:10.419462 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a3ebec0e-c743-4853-8a57-af0620c5eb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:10.442781 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8279f-8720-4e6b-9ebf-9121f90710a8 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:10.444091 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8279f-8720-4e6b-9ebf-9121f90710a8 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:10.444091 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8279f-8720-4e6b-9ebf-9121f90710a8 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:10.446209 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 431/872] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:10.451610 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6af427-497c-449d-ab05-7e20cf29c3b1 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:10.451900 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6af427-497c-449d-ab05-7e20cf29c3b1 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:10.452182 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6af427-497c-449d-ab05-7e20cf29c3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:10.472072 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-2e6af427-497c-449d-ab05-7e20cf29c3b1 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:10.473289 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-2e6af427-497c-449d-ab05-7e20cf29c3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:10.500382 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 442/873] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:10.549512 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 432/874] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:10.778742 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ebec0e-c743-4853-8a57-af0620c5eb77 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:10.824056 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:10.838751 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6af427-497c-449d-ab05-7e20cf29c3b1 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:10.865323 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:10.902024 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ebec0e-c743-4853-8a57-af0620c5eb77 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:10.954452 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26af354-751a-4ab6-b111-dc6c6af95ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:10.955199 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26af354-751a-4ab6-b111-dc6c6af95ea6 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:10.955481 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26af354-751a-4ab6-b111-dc6c6af95ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:10.955752 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26af354-751a-4ab6-b111-dc6c6af95ea6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:10.956339 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26af354-751a-4ab6-b111-dc6c6af95ea6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:10.958272 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26af354-751a-4ab6-b111-dc6c6af95ea6 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:10.971571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-a26af354-751a-4ab6-b111-dc6c6af95ea6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 14:55:10.971571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:10.971571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:55:10.971571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:55:10.971571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:55:10.971571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:55:10.971571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:55:10.971571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:10.971571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 14:55:10.971571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:10.971571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:55:10.971571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:10.971571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:10.971571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:10.971571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:10.971571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:10.971571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:10.972724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:10.972724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:10.972724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:10.972724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:10.972724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:10.972724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:10.972724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:55:10.972724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:55:10.972724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:55:10.972724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:55:10.972724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:10.972724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:10.972724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:10.972724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:10.972724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:10.972724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:10.973670 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:10.973670 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:10.973670 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:10.973670 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:10.973670 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:10.973670 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:10.973670 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:10.973670 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:10.973670 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:10.973670 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:10.973670 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:55:10.973670 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:10.973670 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 14:55:10.973670 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:10.978741 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 443/875] 10.222.0.33 () {56 vars in 1126 bytes} [Sun Sep 11 14:55:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:10.979091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26af354-751a-4ab6-b111-dc6c6af95ea6 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:10.982021 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6af427-497c-449d-ab05-7e20cf29c3b1 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:11.043581 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85497fc7-5f9c-45be-9238-7fae872bec01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:11.044117 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85497fc7-5f9c-45be-9238-7fae872bec01 None admin] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:11.044396 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85497fc7-5f9c-45be-9238-7fae872bec01 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:11.044671 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85497fc7-5f9c-45be-9238-7fae872bec01 None admin] PATH_INFO: `/v3/services` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:11.045397 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85497fc7-5f9c-45be-9238-7fae872bec01 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:11.047045 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85497fc7-5f9c-45be-9238-7fae872bec01 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:11.064236 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 433/876] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 11 14:55:10 2022] POST /identity/v3/services => generated 251 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 14:55:11.065551 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d2290-0836-4628-9b4b-37347e4f75f8 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:11.065833 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d2290-0836-4628-9b4b-37347e4f75f8 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:11.066096 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d2290-0836-4628-9b4b-37347e4f75f8 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:11.070959 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 434/877] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:11.073879 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ac91c-f052-473a-bb91-eb08df5129a3 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:11.074171 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ac91c-f052-473a-bb91-eb08df5129a3 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:11.074446 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ac91c-f052-473a-bb91-eb08df5129a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:11.097414 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-fd9ac91c-f052-473a-bb91-eb08df5129a3 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:11.098696 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-fd9ac91c-f052-473a-bb91-eb08df5129a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:11.123039 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26af354-751a-4ab6-b111-dc6c6af95ea6 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:11.131629 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:11.171749 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b7f03-a320-4f80-9e14-a30cef8e256d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:11.172391 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b7f03-a320-4f80-9e14-a30cef8e256d None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:11.172674 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b7f03-a320-4f80-9e14-a30cef8e256d None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:11.172992 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b7f03-a320-4f80-9e14-a30cef8e256d None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:11.173681 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b7f03-a320-4f80-9e14-a30cef8e256d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:11.174049 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b7f03-a320-4f80-9e14-a30cef8e256d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:11.175613 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b7f03-a320-4f80-9e14-a30cef8e256d None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:11.186627 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 444/878] 10.222.0.33 () {56 vars in 1122 bytes} [Sun Sep 11 14:55:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:55:11.188048 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b7f03-a320-4f80-9e14-a30cef8e256d None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:11.198847 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 435/879] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:11 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:11.307033 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b7f03-a320-4f80-9e14-a30cef8e256d None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:11.346554 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d51cfc-ff84-4557-b660-60c126680ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:11.347613 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d51cfc-ff84-4557-b660-60c126680ef2 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:11.348062 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d51cfc-ff84-4557-b660-60c126680ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:11.348499 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d51cfc-ff84-4557-b660-60c126680ef2 None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:11.349290 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d51cfc-ff84-4557-b660-60c126680ef2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:11.349751 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d51cfc-ff84-4557-b660-60c126680ef2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:11.351339 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d51cfc-ff84-4557-b660-60c126680ef2 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:11.356697 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 445/880] 10.222.0.33 () {56 vars in 1122 bytes} [Sun Sep 11 14:55:11 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:11.488499 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ac91c-f052-473a-bb91-eb08df5129a3 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:11.501679 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:11.601162 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ac91c-f052-473a-bb91-eb08df5129a3 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:11.636449 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e4e04-f659-4937-ba85-c05fb48c98bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:11.637062 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e4e04-f659-4937-ba85-c05fb48c98bf None admin] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:11.637343 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e4e04-f659-4937-ba85-c05fb48c98bf None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:11.637618 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e4e04-f659-4937-ba85-c05fb48c98bf None admin] PATH_INFO: `/v3/services` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:11.638289 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e4e04-f659-4937-ba85-c05fb48c98bf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:11.639767 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e4e04-f659-4937-ba85-c05fb48c98bf None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:11.657834 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 436/881] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 11 14:55:11 2022] POST /identity/v3/services => generated 269 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:55:12.327228 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cba7a9-1d1a-48fc-bf4d-74cc74bc0b44 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:12.327742 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cba7a9-1d1a-48fc-bf4d-74cc74bc0b44 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:12.327742 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cba7a9-1d1a-48fc-bf4d-74cc74bc0b44 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:12.329274 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 446/882] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:12.334605 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f24be-5ab6-4f22-9da9-50b2d85ebdd9 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:12.335080 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f24be-5ab6-4f22-9da9-50b2d85ebdd9 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:12.335168 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f24be-5ab6-4f22-9da9-50b2d85ebdd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:12.354223 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-883f24be-5ab6-4f22-9da9-50b2d85ebdd9 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:12.355458 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-883f24be-5ab6-4f22-9da9-50b2d85ebdd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:12.421994 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:12.432879 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 437/883] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:12 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:12.705644 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cba7a9-1d1a-48fc-bf4d-74cc74bc0b44 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:12.742829 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:12.820610 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cba7a9-1d1a-48fc-bf4d-74cc74bc0b44 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:12.857682 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810df245-5bd8-4561-a48d-eb8617995e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:12.858379 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810df245-5bd8-4561-a48d-eb8617995e13 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:12.858660 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810df245-5bd8-4561-a48d-eb8617995e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:12.859022 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810df245-5bd8-4561-a48d-eb8617995e13 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:12.871480 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810df245-5bd8-4561-a48d-eb8617995e13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:55:12.877642 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-810df245-5bd8-4561-a48d-eb8617995e13 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 14:55:12.877642 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:12.877642 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:12.877642 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:12.877642 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:12.877642 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:12.877642 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:12.877642 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:12.877642 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:12.877642 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:12.877642 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:12.877642 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:12.877642 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:55:12.877642 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:55:12.877642 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:55:12.877642 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:55:12.878562 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:55:12.878562 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:55:12.878562 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:55:12.878562 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:55:12.878562 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:12.878562 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:12.878562 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:12.878562 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:12.878562 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:12.878562 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:12.878562 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:12.878562 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:12.878562 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:12.878562 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:12.878562 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:12.878562 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:12.879462 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:12.879462 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:12.879462 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:12.879462 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:12.879462 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:55:12.879462 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:55:12.879462 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:55:12.879462 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:55:12.879462 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:55:12.879462 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:55:12.879462 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 14:55:12.879462 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:12.879462 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 447/884] 10.222.0.33 () {56 vars in 1120 bytes} [Sun Sep 11 14:55:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:12.884079 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f24be-5ab6-4f22-9da9-50b2d85ebdd9 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:12.995116 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f24be-5ab6-4f22-9da9-50b2d85ebdd9 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:13.037729 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091eda2d-bcbb-4c59-a9a1-5ebf2c4265db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:13.038681 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091eda2d-bcbb-4c59-a9a1-5ebf2c4265db None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:13.039211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091eda2d-bcbb-4c59-a9a1-5ebf2c4265db None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:13.039689 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091eda2d-bcbb-4c59-a9a1-5ebf2c4265db None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:13.040902 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091eda2d-bcbb-4c59-a9a1-5ebf2c4265db None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:13.041458 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091eda2d-bcbb-4c59-a9a1-5ebf2c4265db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:13.044072 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091eda2d-bcbb-4c59-a9a1-5ebf2c4265db None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:13.055072 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 438/885] 10.222.0.33 () {56 vars in 1115 bytes} [Sun Sep 11 14:55:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:13.061979 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810df245-5bd8-4561-a48d-eb8617995e13 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:13.163979 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:13.176504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810df245-5bd8-4561-a48d-eb8617995e13 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:13.212742 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a4790-9234-4e96-8284-d6f84df62241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:13.213418 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a4790-9234-4e96-8284-d6f84df62241 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:13.213708 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a4790-9234-4e96-8284-d6f84df62241 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:13.213986 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a4790-9234-4e96-8284-d6f84df62241 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:13.225553 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2a4790-9234-4e96-8284-d6f84df62241 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:13.227092 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2a4790-9234-4e96-8284-d6f84df62241 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:13.242091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-fb2a4790-9234-4e96-8284-d6f84df62241 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:13.242091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:13.242091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:13.242091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:13.242091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:13.242091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:13.242091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:13.242091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:13.242091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:13.242091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:13.242091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:13.242091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:13.242091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:55:13.242091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:55:13.242091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:55:13.242091 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:55:13.244049 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:13.244049 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:13.244049 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:13.244049 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:13.244049 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:13.244049 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:13.244049 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:13.244049 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:13.244049 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:13.244049 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:13.244049 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:13.244049 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:13.244049 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:13.244049 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:13.244049 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:13.244049 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:13.244049 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:55:13.245266 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:55:13.245266 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:13.245266 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:13.245266 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 448/886] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:55:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:13.250411 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091eda2d-bcbb-4c59-a9a1-5ebf2c4265db None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:13.317215 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:13.364603 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091eda2d-bcbb-4c59-a9a1-5ebf2c4265db None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:13.400065 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde58003-106a-4905-ad73-a072f5955df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:13.400677 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde58003-106a-4905-ad73-a072f5955df8 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:13.400955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde58003-106a-4905-ad73-a072f5955df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:13.401342 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde58003-106a-4905-ad73-a072f5955df8 None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:13.401993 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde58003-106a-4905-ad73-a072f5955df8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:13.402294 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde58003-106a-4905-ad73-a072f5955df8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:13.403709 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde58003-106a-4905-ad73-a072f5955df8 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:13.420493 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 439/887] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:55:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:13.422894 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a4790-9234-4e96-8284-d6f84df62241 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:13.432292 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503a10e-e597-4a7b-8cb8-39c38614dc77 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:13.432292 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503a10e-e597-4a7b-8cb8-39c38614dc77 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:13.432292 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503a10e-e597-4a7b-8cb8-39c38614dc77 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:13.433083 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 440/888] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:55:13.441830 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcd2c0-c03b-4189-8e19-7991accbbfaf None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:13.441830 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcd2c0-c03b-4189-8e19-7991accbbfaf None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:13.441830 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcd2c0-c03b-4189-8e19-7991accbbfaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:13.458217 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-07bcd2c0-c03b-4189-8e19-7991accbbfaf None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:13.459738 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-07bcd2c0-c03b-4189-8e19-7991accbbfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:13.541298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a4790-9234-4e96-8284-d6f84df62241 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:13.541656 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 441/889] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:13 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 14:55:13.585977 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c360e9-8149-40f2-bbad-a34baab363fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:13.586628 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c360e9-8149-40f2-bbad-a34baab363fb None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:13.586995 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c360e9-8149-40f2-bbad-a34baab363fb None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:13.587266 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c360e9-8149-40f2-bbad-a34baab363fb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:13.596535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c360e9-8149-40f2-bbad-a34baab363fb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:13.598013 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c360e9-8149-40f2-bbad-a34baab363fb None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:13.610401 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-16c360e9-8149-40f2-bbad-a34baab363fb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 14:55:13.610401 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:13.610401 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:13.610401 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:13.610401 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:13.610401 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:13.610401 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:13.610401 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:13.610401 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:13.610401 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:13.610401 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:13.610401 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:13.610401 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:55:13.610401 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:55:13.610401 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:55:13.610401 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:55:13.611398 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:13.611398 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:13.611398 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:55:13.611398 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:55:13.611398 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:55:13.611398 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:55:13.611398 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:13.611398 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:13.611398 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:13.611398 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:13.611398 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:13.611398 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:13.611398 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:13.611398 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:13.611398 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:13.611398 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:13.611398 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:13.611398 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:13.612356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:13.612356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:13.612356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:55:13.612356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:55:13.612356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:55:13.612356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:55:13.612356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:55:13.612356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:55:13.612356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 14:55:13.612356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:13.612356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 449/890] 10.222.0.33 () {56 vars in 1117 bytes} [Sun Sep 11 14:55:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:13.617329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bcd2c0-c03b-4189-8e19-7991accbbfaf None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:13.687175 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:13.688579 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c9a41-7d1d-4c41-babe-08ace0bc4d65 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:13.688842 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c9a41-7d1d-4c41-babe-08ace0bc4d65 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:13.689120 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c9a41-7d1d-4c41-babe-08ace0bc4d65 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:13.690637 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 450/891] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:13.700198 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10689c-36d4-421f-8ae2-e024488d1408 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:13.700818 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10689c-36d4-421f-8ae2-e024488d1408 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:13.701849 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10689c-36d4-421f-8ae2-e024488d1408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:13.708733 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:13.723233 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-7c10689c-36d4-421f-8ae2-e024488d1408 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:13.724781 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7c10689c-36d4-421f-8ae2-e024488d1408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:13.725773 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bcd2c0-c03b-4189-8e19-7991accbbfaf None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:13.764649 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c38da-9505-40b9-bc41-098a0d08db7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:13.765712 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c38da-9505-40b9-bc41-098a0d08db7c None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:13.766259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c38da-9505-40b9-bc41-098a0d08db7c None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:13.766678 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c38da-9505-40b9-bc41-098a0d08db7c None admin] PATH_INFO: `/v3/users` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:13.767627 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37c38da-9505-40b9-bc41-098a0d08db7c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:13.768111 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37c38da-9505-40b9-bc41-098a0d08db7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:13.769756 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37c38da-9505-40b9-bc41-098a0d08db7c None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:13.795501 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 442/892] 10.222.0.33 () {56 vars in 1149 bytes} [Sun Sep 11 14:55:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:13.797560 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c38da-9505-40b9-bc41-098a0d08db7c None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:13.802493 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 451/893] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:13 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:13.827925 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c10689c-36d4-421f-8ae2-e024488d1408 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:13.905129 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c38da-9505-40b9-bc41-098a0d08db7c None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:13.913010 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:13.945330 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a513e-5457-4137-b366-239fde683c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:13.945813 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c10689c-36d4-421f-8ae2-e024488d1408 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:13.946243 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a513e-5457-4137-b366-239fde683c8a None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:13.946586 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a513e-5457-4137-b366-239fde683c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:13.946890 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a513e-5457-4137-b366-239fde683c8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:13.979239 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:13.986131 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d17b39a-af4e-421f-bde6-07f325c0481f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:13.987059 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17b39a-af4e-421f-bde6-07f325c0481f None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:13.987663 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17b39a-af4e-421f-bde6-07f325c0481f None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:13.988116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17b39a-af4e-421f-bde6-07f325c0481f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:13.989656 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d17b39a-af4e-421f-bde6-07f325c0481f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:13.991258 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d17b39a-af4e-421f-bde6-07f325c0481f None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:14.002298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-0d17b39a-af4e-421f-bde6-07f325c0481f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 14:55:14.002298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:14.002298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:55:14.002298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:55:14.002298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:55:14.002298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:55:14.002298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:55:14.002298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:14.002298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 14:55:14.002298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:14.002298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:55:14.002298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:14.002298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:14.002298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:14.002298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:14.002298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:14.002298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:14.003307 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:14.003307 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:14.003307 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:14.003307 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:14.003307 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:14.003307 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:14.003307 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:55:14.003307 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:55:14.003307 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:55:14.003307 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:55:14.003307 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:14.003307 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:14.003307 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:14.003307 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:14.003307 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:14.003307 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:14.004180 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:14.004180 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:14.004180 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:14.004180 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:14.004180 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:14.004180 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:14.004180 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:14.004180 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:14.004180 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:14.004180 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:14.004180 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:55:14.004180 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:14.004180 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 14:55:14.004180 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:14.004180 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 452/894] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 14:55:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:14.005606 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac278a57-ba0b-4854-aec8-b533a148d9fc None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:14.005606 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac278a57-ba0b-4854-aec8-b533a148d9fc None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:14.005606 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac278a57-ba0b-4854-aec8-b533a148d9fc None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:14.006514 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 453/895] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:14.011121 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac278a57-ba0b-4854-aec8-b533a148d9fc None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:14.028501 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a513e-5457-4137-b366-239fde683c8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:14.030056 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a513e-5457-4137-b366-239fde683c8a None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:14.119951 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac278a57-ba0b-4854-aec8-b533a148d9fc None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:14.136890 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 443/896] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:55:13 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 14:55:14.138165 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab463b5f-fe45-48a1-9784-07e1d69ac969 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:14.139036 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab463b5f-fe45-48a1-9784-07e1d69ac969 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:14.139394 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab463b5f-fe45-48a1-9784-07e1d69ac969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:14.151166 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:14.154846 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52778b3a-96e5-487b-bd5a-3f2028a0d3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:14.155465 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52778b3a-96e5-487b-bd5a-3f2028a0d3ce None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:14.155741 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52778b3a-96e5-487b-bd5a-3f2028a0d3ce None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:14.156012 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52778b3a-96e5-487b-bd5a-3f2028a0d3ce None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:14.156644 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52778b3a-96e5-487b-bd5a-3f2028a0d3ce None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:14.156949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52778b3a-96e5-487b-bd5a-3f2028a0d3ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:14.157644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-ab463b5f-fe45-48a1-9784-07e1d69ac969 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:14.158361 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52778b3a-96e5-487b-bd5a-3f2028a0d3ce None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:14.158856 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-ab463b5f-fe45-48a1-9784-07e1d69ac969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:14.163001 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 454/897] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 14:55:14 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:14.163757 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52778b3a-96e5-487b-bd5a-3f2028a0d3ce None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:14.224411 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:14.234846 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 444/898] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:14 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:14.234846 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab463b5f-fe45-48a1-9784-07e1d69ac969 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:14.286630 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52778b3a-96e5-487b-bd5a-3f2028a0d3ce None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:14.322141 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8bd9a-3017-49fc-851c-5b2f5d8dd3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:14.322747 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8bd9a-3017-49fc-851c-5b2f5d8dd3db None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:14.323056 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8bd9a-3017-49fc-851c-5b2f5d8dd3db None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:14.323326 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8bd9a-3017-49fc-851c-5b2f5d8dd3db None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:14.323969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8bd9a-3017-49fc-851c-5b2f5d8dd3db None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:14.325336 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8bd9a-3017-49fc-851c-5b2f5d8dd3db None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:14.333843 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab463b5f-fe45-48a1-9784-07e1d69ac969 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:14.342659 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 455/899] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 11 14:55:14 2022] POST /identity/v3/services => generated 241 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:55:14.343594 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8bd9a-3017-49fc-851c-5b2f5d8dd3db None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:14.396512 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:14.405236 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ff4e6-11b1-4c9d-8831-3e509925ada2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:14.406239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ff4e6-11b1-4c9d-8831-3e509925ada2 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:14.407240 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ff4e6-11b1-4c9d-8831-3e509925ada2 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:14.407703 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ff4e6-11b1-4c9d-8831-3e509925ada2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:14.423811 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3ff4e6-11b1-4c9d-8831-3e509925ada2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:14.426129 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3ff4e6-11b1-4c9d-8831-3e509925ada2 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:14.448986 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-8e3ff4e6-11b1-4c9d-8831-3e509925ada2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:14.448986 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:14.448986 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:14.448986 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:14.448986 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:14.448986 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:14.448986 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:14.448986 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:14.448986 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:14.448986 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:14.448986 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:14.448986 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:14.448986 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:55:14.448986 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:55:14.448986 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:55:14.448986 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:55:14.450039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:14.450039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:14.450039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:14.450039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:14.450039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:14.450039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:14.450039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:14.450039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:14.450039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:14.450039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:14.450039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:14.450039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:14.450039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:14.450039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:14.450039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:14.450039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:14.450039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:55:14.451073 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:55:14.451073 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:14.451073 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:14.457631 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 445/900] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:55:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:14.459894 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ff4e6-11b1-4c9d-8831-3e509925ada2 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:14.478647 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8bd9a-3017-49fc-851c-5b2f5d8dd3db None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:14.503345 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:14.522424 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b3642-ab15-4a42-9433-6f5921450465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:14.523258 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b3642-ab15-4a42-9433-6f5921450465 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:14.523399 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b3642-ab15-4a42-9433-6f5921450465 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:14.523602 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b3642-ab15-4a42-9433-6f5921450465 None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:14.524239 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279b3642-ab15-4a42-9433-6f5921450465 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:14.524537 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279b3642-ab15-4a42-9433-6f5921450465 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:14.525920 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279b3642-ab15-4a42-9433-6f5921450465 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:14.534985 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 456/901] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 14:55:14 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:14.535830 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b3642-ab15-4a42-9433-6f5921450465 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:14.581358 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ff4e6-11b1-4c9d-8831-3e509925ada2 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:14.615416 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46987178-d6e6-422f-b063-32fa3530a7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:14.616025 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46987178-d6e6-422f-b063-32fa3530a7b6 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:14.616300 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46987178-d6e6-422f-b063-32fa3530a7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:14.616570 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46987178-d6e6-422f-b063-32fa3530a7b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:14.617204 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46987178-d6e6-422f-b063-32fa3530a7b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:14.617502 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46987178-d6e6-422f-b063-32fa3530a7b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:14.618892 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46987178-d6e6-422f-b063-32fa3530a7b6 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:14.635579 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 446/902] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:55:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:14.635579 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46987178-d6e6-422f-b063-32fa3530a7b6 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:14.644608 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b3642-ab15-4a42-9433-6f5921450465 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:14.680384 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d98c8f-cb52-4c6d-ad17-55262622cd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:14.680947 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d98c8f-cb52-4c6d-ad17-55262622cd15 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:14.681226 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d98c8f-cb52-4c6d-ad17-55262622cd15 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:14.681500 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d98c8f-cb52-4c6d-ad17-55262622cd15 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:14.682084 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d98c8f-cb52-4c6d-ad17-55262622cd15 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:14.683750 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d98c8f-cb52-4c6d-ad17-55262622cd15 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:14.688505 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:14.692950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-46d98c8f-cb52-4c6d-ad17-55262622cd15 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 14:55:14.692950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:14.692950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:55:14.692950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:55:14.692950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:55:14.692950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:55:14.692950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:55:14.692950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:14.692950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 14:55:14.692950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:14.692950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:55:14.692950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:14.692950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:14.692950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:14.692950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:14.692950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:14.693873 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:14.693873 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:14.693873 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:14.693873 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:14.693873 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:14.693873 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:14.693873 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:14.693873 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:55:14.693873 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:55:14.693873 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:55:14.693873 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:55:14.693873 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:14.693873 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:14.693873 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:14.693873 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:14.693873 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:14.693873 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:14.694784 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:14.694784 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:14.694784 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:14.694784 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:14.694784 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:14.694784 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:14.694784 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:14.694784 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:14.694784 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:14.694784 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:14.694784 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:55:14.694784 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:14.694784 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 14:55:14.694784 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:14.694784 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 457/903] 10.222.0.33 () {56 vars in 1132 bytes} [Sun Sep 11 14:55:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:14.695627 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d98c8f-cb52-4c6d-ad17-55262622cd15 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:14.746728 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:14.755320 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46987178-d6e6-422f-b063-32fa3530a7b6 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:14.792973 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a552b-c769-43bf-a0bc-d800b609a984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:14.793683 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a552b-c769-43bf-a0bc-d800b609a984 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:14.793958 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a552b-c769-43bf-a0bc-d800b609a984 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:14.794339 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a552b-c769-43bf-a0bc-d800b609a984 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:14.795654 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16a552b-c769-43bf-a0bc-d800b609a984 None admin] RBAC: Adding query filter params (interface=public, service_id=411b19760e154a42bb922daa3b5152ba, region_id=RegionOne) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:14.795960 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16a552b-c769-43bf-a0bc-d800b609a984 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:14.797383 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16a552b-c769-43bf-a0bc-d800b609a984 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:14.798026 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d98c8f-cb52-4c6d-ad17-55262622cd15 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:14.805161 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 447/904] 10.222.0.33 () {56 vars in 1251 bytes} [Sun Sep 11 14:55:14 2022] GET /identity/v3/endpoints?service_id=411b19760e154a42bb922daa3b5152ba&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:14.806010 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a552b-c769-43bf-a0bc-d800b609a984 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:14.837094 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22edabfe-1179-4f63-974c-25dc5af2f881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:14.837712 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22edabfe-1179-4f63-974c-25dc5af2f881 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:14.837994 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22edabfe-1179-4f63-974c-25dc5af2f881 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:14.838271 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22edabfe-1179-4f63-974c-25dc5af2f881 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:14.845727 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22edabfe-1179-4f63-974c-25dc5af2f881 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:14.847222 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22edabfe-1179-4f63-974c-25dc5af2f881 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:14.858638 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-22edabfe-1179-4f63-974c-25dc5af2f881 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:55:14.858638 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:14.858638 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:14.858638 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:14.858638 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:14.858638 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:14.858638 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:14.858638 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:14.858638 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:14.858638 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:14.858638 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:14.858638 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:14.858638 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:55:14.858638 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:55:14.858638 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:55:14.859969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:55:14.859969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:14.859969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:14.859969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:14.859969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:14.859969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:14.859969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:14.859969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:14.859969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:14.859969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:14.859969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:14.859969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:14.859969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:14.859969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:14.859969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:14.859969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:14.859969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:14.861277 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:55:14.861277 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:55:14.861277 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:55:14.861277 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:55:14.861277 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:55:14.861277 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:55:14.861277 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:55:14.861277 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:14.864914 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 458/905] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:55:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:14.865591 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22edabfe-1179-4f63-974c-25dc5af2f881 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:14.921509 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a552b-c769-43bf-a0bc-d800b609a984 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:14.930918 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:14.959659 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c92aca-4979-4822-bd47-3d736ce139fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:14.960267 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c92aca-4979-4822-bd47-3d736ce139fc None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:14.960584 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c92aca-4979-4822-bd47-3d736ce139fc None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:14.960819 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c92aca-4979-4822-bd47-3d736ce139fc None admin] PATH_INFO: `/v3/services` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:14.961444 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c92aca-4979-4822-bd47-3d736ce139fc None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:14.961747 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c92aca-4979-4822-bd47-3d736ce139fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:14.963165 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c92aca-4979-4822-bd47-3d736ce139fc None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:14.967746 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22edabfe-1179-4f63-974c-25dc5af2f881 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:14.968804 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 448/906] 10.222.0.33 () {56 vars in 1126 bytes} [Sun Sep 11 14:55:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:14.969389 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c92aca-4979-4822-bd47-3d736ce139fc None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:14.981719 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:15.017273 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac5e2c-c0ee-437a-81f8-34f131fd61bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:15.017918 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac5e2c-c0ee-437a-81f8-34f131fd61bf None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:15.018197 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac5e2c-c0ee-437a-81f8-34f131fd61bf None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:15.018472 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac5e2c-c0ee-437a-81f8-34f131fd61bf None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:15.019121 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac5e2c-c0ee-437a-81f8-34f131fd61bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:15.020638 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac5e2c-c0ee-437a-81f8-34f131fd61bf None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:15.030374 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 459/907] 10.222.0.33 () {56 vars in 1087 bytes} [Sun Sep 11 14:55:14 2022] GET /identity/v3/services => generated 1497 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 14:55:15.031333 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac5e2c-c0ee-437a-81f8-34f131fd61bf None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:15.115369 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c92aca-4979-4822-bd47-3d736ce139fc None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:15.159404 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac5e2c-c0ee-437a-81f8-34f131fd61bf None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:15.164151 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709918f5-3b69-4a88-878d-15f89502e584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:15.165106 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709918f5-3b69-4a88-878d-15f89502e584 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:15.165632 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709918f5-3b69-4a88-878d-15f89502e584 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:15.166111 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709918f5-3b69-4a88-878d-15f89502e584 None admin] PATH_INFO: `/v3/projects` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:15.167057 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709918f5-3b69-4a88-878d-15f89502e584 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:15.167660 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709918f5-3b69-4a88-878d-15f89502e584 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:15.169401 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709918f5-3b69-4a88-878d-15f89502e584 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:15.185969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 449/908] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:55:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:15.193241 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709918f5-3b69-4a88-878d-15f89502e584 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:15.207367 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7644be-469c-43bc-b092-63ad423e397c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:15.208054 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7644be-469c-43bc-b092-63ad423e397c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:15.208343 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7644be-469c-43bc-b092-63ad423e397c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:15.208646 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7644be-469c-43bc-b092-63ad423e397c None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:15.210004 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7644be-469c-43bc-b092-63ad423e397c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:15.210324 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7644be-469c-43bc-b092-63ad423e397c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:15.212010 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7644be-469c-43bc-b092-63ad423e397c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:15.220178 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 460/909] 10.222.0.33 () {56 vars in 1126 bytes} [Sun Sep 11 14:55:15 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:15.226825 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7644be-469c-43bc-b092-63ad423e397c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:15.237819 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:15.275612 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:15.328226 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709918f5-3b69-4a88-878d-15f89502e584 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:15.349340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7644be-469c-43bc-b092-63ad423e397c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:15.374161 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8c477-785f-4f87-8b35-0317bb09319e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:15.375264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8c477-785f-4f87-8b35-0317bb09319e None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:15.375713 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8c477-785f-4f87-8b35-0317bb09319e None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:15.376291 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8c477-785f-4f87-8b35-0317bb09319e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:15.377505 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8c477-785f-4f87-8b35-0317bb09319e None admin] RBAC: Adding query filter params (role.id=56ed900824b74d938089a39aeb349cf1, scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176, user.id=0aafb2bbba1d466fb22bb0824a531ddc) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:15.378235 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8c477-785f-4f87-8b35-0317bb09319e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:15.380333 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8c477-785f-4f87-8b35-0317bb09319e None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:15.385321 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72da15d-4600-4cfd-bae9-c4f91c2601b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:15.386041 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72da15d-4600-4cfd-bae9-c4f91c2601b3 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:15.386423 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72da15d-4600-4cfd-bae9-c4f91c2601b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:15.386896 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72da15d-4600-4cfd-bae9-c4f91c2601b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:15.387739 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72da15d-4600-4cfd-bae9-c4f91c2601b3 None admin] RBAC: Adding query filter params (interface=public, service_id=edd7dd726cce4e9ca4d9a034ef94ae02, region_id=RegionOne) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:15.387900 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 450/910] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:55:15 2022] GET /identity/v3/role_assignments?user.id=0aafb2bbba1d466fb22bb0824a531ddc&scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176&role.id=56ed900824b74d938089a39aeb349cf1 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:15.388250 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72da15d-4600-4cfd-bae9-c4f91c2601b3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:15.389814 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72da15d-4600-4cfd-bae9-c4f91c2601b3 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:15.399986 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 461/911] 10.222.0.33 () {56 vars in 1251 bytes} [Sun Sep 11 14:55:15 2022] GET /identity/v3/endpoints?service_id=edd7dd726cce4e9ca4d9a034ef94ae02&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:55:15.405852 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8c477-785f-4f87-8b35-0317bb09319e None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:15.517221 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:15.523885 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8c477-785f-4f87-8b35-0317bb09319e None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:15.567866 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cebee5-6e78-4b26-a64d-62a8c72a2b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:15.568498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cebee5-6e78-4b26-a64d-62a8c72a2b5b None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:15.568778 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cebee5-6e78-4b26-a64d-62a8c72a2b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:15.569052 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cebee5-6e78-4b26-a64d-62a8c72a2b5b None admin] PATH_INFO: `/v3/services` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:15.569678 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cebee5-6e78-4b26-a64d-62a8c72a2b5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:15.571155 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cebee5-6e78-4b26-a64d-62a8c72a2b5b None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:15.578222 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 451/912] 10.222.0.33 () {56 vars in 1087 bytes} [Sun Sep 11 14:55:15 2022] GET /identity/v3/services => generated 1497 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 14:55:16.530334 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde5c7a-a552-449d-96e5-8e919057fee1 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:16.530891 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde5c7a-a552-449d-96e5-8e919057fee1 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:16.530891 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde5c7a-a552-449d-96e5-8e919057fee1 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:16.532491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 462/913] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:16.537840 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f45ab-c601-4d75-85cb-6c3f05e2c55b None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:16.538229 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f45ab-c601-4d75-85cb-6c3f05e2c55b None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:16.538409 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f45ab-c601-4d75-85cb-6c3f05e2c55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:16.558179 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-e18f45ab-c601-4d75-85cb-6c3f05e2c55b None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:16.560162 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-e18f45ab-c601-4d75-85cb-6c3f05e2c55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:16.631518 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 452/914] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:16.896846 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bde5c7a-a552-449d-96e5-8e919057fee1 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:17.054829 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:17.059111 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bde5c7a-a552-449d-96e5-8e919057fee1 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:17.118848 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7b082-391f-4b1c-9bbf-8f53c2d15f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:17.122846 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7b082-391f-4b1c-9bbf-8f53c2d15f1b None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:17.122846 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7b082-391f-4b1c-9bbf-8f53c2d15f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:17.122846 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7b082-391f-4b1c-9bbf-8f53c2d15f1b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:17.122846 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b7b082-391f-4b1c-9bbf-8f53c2d15f1b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:17.122846 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b7b082-391f-4b1c-9bbf-8f53c2d15f1b None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:17.155390 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-15b7b082-391f-4b1c-9bbf-8f53c2d15f1b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 14:55:17.155390 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:17.155390 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:55:17.155390 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:55:17.155390 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:55:17.155390 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:55:17.155390 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:55:17.155390 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:17.155390 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 14:55:17.155390 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:17.155390 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:55:17.155390 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:17.155390 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:17.155390 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:17.155390 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:17.155390 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:17.155390 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:17.156381 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:17.156381 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:17.156381 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:17.156381 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:17.156381 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:17.156381 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:17.156381 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:55:17.156381 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:55:17.156381 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:55:17.156381 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:55:17.156381 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:17.156381 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:17.156381 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:17.156381 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:17.156381 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:17.156381 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:17.157213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:17.157213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:17.157213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:17.157213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:17.157213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:17.157213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:17.157213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:17.157213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:17.157213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:17.157213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:17.157213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:55:17.157213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:17.157213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 14:55:17.157213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:17.160036 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 463/915] 10.222.0.33 () {56 vars in 1126 bytes} [Sun Sep 11 14:55:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:17.166708 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f45ab-c601-4d75-85cb-6c3f05e2c55b None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:17.284702 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:17.297154 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f45ab-c601-4d75-85cb-6c3f05e2c55b None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:17.338838 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a5490-a017-463b-8b08-286fe1f6f123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:17.338838 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a5490-a017-463b-8b08-286fe1f6f123 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:17.338838 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a5490-a017-463b-8b08-286fe1f6f123 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:17.338838 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a5490-a017-463b-8b08-286fe1f6f123 None admin] PATH_INFO: `/v3/services` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:17.339262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a5490-a017-463b-8b08-286fe1f6f123 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:17.339262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a5490-a017-463b-8b08-286fe1f6f123 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:17.339583 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a5490-a017-463b-8b08-286fe1f6f123 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:17.346161 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 453/916] 10.222.0.33 () {56 vars in 1122 bytes} [Sun Sep 11 14:55:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:17.351819 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7b082-391f-4b1c-9bbf-8f53c2d15f1b None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:17.509359 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b154c-dc69-4d59-81c2-65e6fdf031c2 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:17.509710 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b154c-dc69-4d59-81c2-65e6fdf031c2 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:17.510012 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b154c-dc69-4d59-81c2-65e6fdf031c2 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:17.511651 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 454/917] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:17.517255 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817701ce-3589-44e1-98f6-237a66254b1e None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:17.517540 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817701ce-3589-44e1-98f6-237a66254b1e None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:17.517818 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817701ce-3589-44e1-98f6-237a66254b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:17.555651 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-817701ce-3589-44e1-98f6-237a66254b1e None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:17.557521 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-817701ce-3589-44e1-98f6-237a66254b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:17.578840 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7b082-391f-4b1c-9bbf-8f53c2d15f1b None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:17.662884 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd423ec-8f9c-430e-86c0-f0d89f759611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:17.667469 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd423ec-8f9c-430e-86c0-f0d89f759611 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:17.667918 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd423ec-8f9c-430e-86c0-f0d89f759611 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:17.668326 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd423ec-8f9c-430e-86c0-f0d89f759611 None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:17.669119 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd423ec-8f9c-430e-86c0-f0d89f759611 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:17.671304 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd423ec-8f9c-430e-86c0-f0d89f759611 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:17.672870 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd423ec-8f9c-430e-86c0-f0d89f759611 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:17.691201 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 464/918] 10.222.0.33 () {56 vars in 1122 bytes} [Sun Sep 11 14:55:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:17.696713 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd423ec-8f9c-430e-86c0-f0d89f759611 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:17.719401 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 455/919] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:17.721089 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b176241-ce86-4c4f-ab4f-63a121d3ec47 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:17.722417 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b176241-ce86-4c4f-ab4f-63a121d3ec47 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:17.722935 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b176241-ce86-4c4f-ab4f-63a121d3ec47 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:17.724646 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 456/920] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:17.729965 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a896fb-1a22-4406-9640-24492ddb6b38 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:17.729965 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a896fb-1a22-4406-9640-24492ddb6b38 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:17.730755 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a896fb-1a22-4406-9640-24492ddb6b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:17.743320 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:17.790966 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-24a896fb-1a22-4406-9640-24492ddb6b38 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:17.792430 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-24a896fb-1a22-4406-9640-24492ddb6b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:17.861947 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd423ec-8f9c-430e-86c0-f0d89f759611 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:17.865777 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 457/921] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 14:55:17.893387 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f96b8a-4c18-444d-a4b8-fd777bb677a6 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:17.894072 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f96b8a-4c18-444d-a4b8-fd777bb677a6 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:17.894548 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f96b8a-4c18-444d-a4b8-fd777bb677a6 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:17.896450 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 458/922] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:55:17.901215 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136fa3a-ec4d-45a6-a930-5c7f48c0c309 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:17.901497 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136fa3a-ec4d-45a6-a930-5c7f48c0c309 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:17.901774 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136fa3a-ec4d-45a6-a930-5c7f48c0c309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:17.903089 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec458363-b2c0-4476-89f0-bcf73c1ca249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:17.903920 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec458363-b2c0-4476-89f0-bcf73c1ca249 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:17.904991 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec458363-b2c0-4476-89f0-bcf73c1ca249 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:17.905270 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec458363-b2c0-4476-89f0-bcf73c1ca249 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:17.905980 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec458363-b2c0-4476-89f0-bcf73c1ca249 None admin] RBAC: Adding query filter params (interface=public, service_id=06804c3cbb45436d8341fe2633c2fe40, region_id=RegionOne) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:17.906285 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec458363-b2c0-4476-89f0-bcf73c1ca249 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:17.907694 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec458363-b2c0-4476-89f0-bcf73c1ca249 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:17.917431 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 465/923] 10.222.0.33 () {56 vars in 1251 bytes} [Sun Sep 11 14:55:17 2022] GET /identity/v3/endpoints?service_id=06804c3cbb45436d8341fe2633c2fe40&interface=public®ion_id=RegionOne => generated 194 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:55:17.919095 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec458363-b2c0-4476-89f0-bcf73c1ca249 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:17.927182 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-8136fa3a-ec4d-45a6-a930-5c7f48c0c309 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:17.928656 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-8136fa3a-ec4d-45a6-a930-5c7f48c0c309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:17.942572 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:18.029909 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 459/924] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:18.031397 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136fa3a-ec4d-45a6-a930-5c7f48c0c309 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:18.038872 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:18.054184 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec458363-b2c0-4476-89f0-bcf73c1ca249 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:18.089327 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653c533e-5f82-4675-bfd5-a8287e43f0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:18.089935 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c533e-5f82-4675-bfd5-a8287e43f0d2 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:18.090218 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c533e-5f82-4675-bfd5-a8287e43f0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:18.090489 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c533e-5f82-4675-bfd5-a8287e43f0d2 None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:18.091137 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653c533e-5f82-4675-bfd5-a8287e43f0d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:18.093406 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653c533e-5f82-4675-bfd5-a8287e43f0d2 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:18.104453 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 466/925] 10.222.0.33 () {56 vars in 1087 bytes} [Sun Sep 11 14:55:17 2022] GET /identity/v3/services => generated 1497 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 14:55:18.136234 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653c533e-5f82-4675-bfd5-a8287e43f0d2 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:18.170987 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136fa3a-ec4d-45a6-a930-5c7f48c0c309 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:18.223978 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e515b2b3-219f-47b8-8341-0a0cf3a1099f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:18.225643 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515b2b3-219f-47b8-8341-0a0cf3a1099f None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:18.226104 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515b2b3-219f-47b8-8341-0a0cf3a1099f None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:18.226550 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515b2b3-219f-47b8-8341-0a0cf3a1099f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:18.227851 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e515b2b3-219f-47b8-8341-0a0cf3a1099f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:18.230204 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e515b2b3-219f-47b8-8341-0a0cf3a1099f None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:18.243644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-e515b2b3-219f-47b8-8341-0a0cf3a1099f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 14:55:18.243644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:18.243644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:55:18.243644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:55:18.243644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:55:18.243644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:55:18.243644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:55:18.243644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:18.243644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 14:55:18.243644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:18.243644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:55:18.243644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:18.243644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:18.243644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:18.243644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:18.243644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:18.243644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:18.244727 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:18.244727 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:18.244727 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:18.244727 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:18.244727 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:18.244727 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:18.244727 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:55:18.244727 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:55:18.244727 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:55:18.244727 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:55:18.244727 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:18.244727 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:18.244727 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:18.244727 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:18.244727 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:18.244727 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:18.245661 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:18.245661 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:18.245661 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:18.245661 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:18.245661 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:18.245661 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:18.245661 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:18.245661 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:18.245661 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:18.245661 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:18.245661 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:55:18.245661 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:18.245661 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 14:55:18.245661 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:18.250248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 460/926] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 14:55:18 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:18.256814 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e515b2b3-219f-47b8-8341-0a0cf3a1099f None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:18.261909 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653c533e-5f82-4675-bfd5-a8287e43f0d2 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:18.268776 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:18.307278 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2294b4df-28c8-4ba0-87e7-48ce6504b33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:18.307841 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2294b4df-28c8-4ba0-87e7-48ce6504b33a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:18.308125 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2294b4df-28c8-4ba0-87e7-48ce6504b33a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:18.308680 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2294b4df-28c8-4ba0-87e7-48ce6504b33a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:18.320408 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2294b4df-28c8-4ba0-87e7-48ce6504b33a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:55:18.325738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-2294b4df-28c8-4ba0-87e7-48ce6504b33a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 14:55:18.325738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:18.325738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:18.325738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:18.325738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:18.325738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:18.325738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:18.325738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:18.325738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:18.325738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:18.325738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:18.325738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:18.325738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:55:18.325738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:55:18.325738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:55:18.325738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:55:18.327256 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:55:18.327256 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:55:18.327256 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:55:18.327256 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:55:18.327256 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:18.327256 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:18.327256 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:18.327256 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:18.327256 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:18.327256 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:18.327256 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:18.327256 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:18.327256 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:18.327256 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:18.327256 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:18.327256 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:18.328319 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:18.328319 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:18.328319 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:18.328319 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:18.328319 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:55:18.328319 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:55:18.328319 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:55:18.328319 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:55:18.328319 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:55:18.328319 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:55:18.328319 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 14:55:18.328319 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:18.328319 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 467/927] 10.222.0.33 () {56 vars in 1120 bytes} [Sun Sep 11 14:55:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:55:18.328319 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2294b4df-28c8-4ba0-87e7-48ce6504b33a None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:18.352267 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:18.370035 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e515b2b3-219f-47b8-8341-0a0cf3a1099f None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:18.409965 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075dbf13-98ed-44e5-b26d-940029b91fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:18.410583 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075dbf13-98ed-44e5-b26d-940029b91fce None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:18.410902 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075dbf13-98ed-44e5-b26d-940029b91fce None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:18.411152 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075dbf13-98ed-44e5-b26d-940029b91fce None admin] PATH_INFO: `/v3/services` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:18.412178 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075dbf13-98ed-44e5-b26d-940029b91fce None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:18.412178 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075dbf13-98ed-44e5-b26d-940029b91fce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:18.413434 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075dbf13-98ed-44e5-b26d-940029b91fce None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:18.422842 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 461/928] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 14:55:18 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:18.422842 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075dbf13-98ed-44e5-b26d-940029b91fce None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:18.446338 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2294b4df-28c8-4ba0-87e7-48ce6504b33a None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:18.480751 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01992402-37af-401f-b7f6-13edcdde3392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:18.481387 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01992402-37af-401f-b7f6-13edcdde3392 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:18.481665 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01992402-37af-401f-b7f6-13edcdde3392 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:18.481932 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01992402-37af-401f-b7f6-13edcdde3392 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:18.482532 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01992402-37af-401f-b7f6-13edcdde3392 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:18.483941 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01992402-37af-401f-b7f6-13edcdde3392 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:18.492566 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-01992402-37af-401f-b7f6-13edcdde3392 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 14:55:18.492566 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:18.492566 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:55:18.492566 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:55:18.492566 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:55:18.492566 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:55:18.492566 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:55:18.492566 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:18.492566 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 14:55:18.492566 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:18.492566 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:55:18.492566 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:18.492566 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:18.492566 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:18.492566 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:18.492566 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:18.493531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:18.493531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:18.493531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:18.493531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:18.493531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:18.493531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:18.493531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:18.493531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:55:18.493531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:55:18.493531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:55:18.493531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:55:18.493531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:18.493531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:18.493531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:18.493531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:18.493531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:18.493531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:18.494435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:18.494435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:18.494435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:18.494435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:18.494435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:18.494435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:18.494435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:18.494435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:18.494435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:18.494435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:18.494435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:55:18.494435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:18.494435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 14:55:18.494435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:18.494435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 468/929] 10.222.0.33 () {56 vars in 1132 bytes} [Sun Sep 11 14:55:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:18.495317 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01992402-37af-401f-b7f6-13edcdde3392 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:18.509019 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:18.543020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075dbf13-98ed-44e5-b26d-940029b91fce None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:18.581961 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea139be3-8cdd-485d-92f3-e28c25b2fa89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:18.583352 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea139be3-8cdd-485d-92f3-e28c25b2fa89 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:18.583812 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea139be3-8cdd-485d-92f3-e28c25b2fa89 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:18.584268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea139be3-8cdd-485d-92f3-e28c25b2fa89 None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:18.585112 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea139be3-8cdd-485d-92f3-e28c25b2fa89 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:18.585597 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea139be3-8cdd-485d-92f3-e28c25b2fa89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:18.586457 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:18.587673 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea139be3-8cdd-485d-92f3-e28c25b2fa89 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:18.597943 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 462/930] 10.222.0.33 () {56 vars in 1115 bytes} [Sun Sep 11 14:55:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:18.598766 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea139be3-8cdd-485d-92f3-e28c25b2fa89 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:18.613814 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01992402-37af-401f-b7f6-13edcdde3392 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:18.654095 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cfc9d1-c396-410e-b2bd-e1333946bf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:18.654973 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cfc9d1-c396-410e-b2bd-e1333946bf1a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:18.655254 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cfc9d1-c396-410e-b2bd-e1333946bf1a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:18.655525 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cfc9d1-c396-410e-b2bd-e1333946bf1a None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:18.656155 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cfc9d1-c396-410e-b2bd-e1333946bf1a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:18.656453 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cfc9d1-c396-410e-b2bd-e1333946bf1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:18.658047 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cfc9d1-c396-410e-b2bd-e1333946bf1a None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:18.668023 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 469/931] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 14:55:18 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:18.668023 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cfc9d1-c396-410e-b2bd-e1333946bf1a None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:18.709508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea139be3-8cdd-485d-92f3-e28c25b2fa89 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:18.749539 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad5acf-d854-4f37-99f5-fc2e38fd06b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:18.750157 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad5acf-d854-4f37-99f5-fc2e38fd06b6 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:18.750450 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad5acf-d854-4f37-99f5-fc2e38fd06b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:18.750727 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad5acf-d854-4f37-99f5-fc2e38fd06b6 None admin] PATH_INFO: `/v3/services` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:18.751364 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cad5acf-d854-4f37-99f5-fc2e38fd06b6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:18.751665 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cad5acf-d854-4f37-99f5-fc2e38fd06b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:18.753039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cad5acf-d854-4f37-99f5-fc2e38fd06b6 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:18.754150 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:18.759322 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 463/932] 10.222.0.33 () {56 vars in 1126 bytes} [Sun Sep 11 14:55:18 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:18.760489 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad5acf-d854-4f37-99f5-fc2e38fd06b6 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:18.774262 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cfc9d1-c396-410e-b2bd-e1333946bf1a None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:18.810128 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae081f5-4e33-4596-a668-d091df9da5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:18.811369 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae081f5-4e33-4596-a668-d091df9da5cc None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:18.811659 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae081f5-4e33-4596-a668-d091df9da5cc None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:18.811933 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae081f5-4e33-4596-a668-d091df9da5cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:18.822852 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae081f5-4e33-4596-a668-d091df9da5cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:18.824322 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae081f5-4e33-4596-a668-d091df9da5cc None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:18.827999 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:18.842585 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-3ae081f5-4e33-4596-a668-d091df9da5cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:18.842585 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:18.842585 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:18.842585 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:18.842585 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:18.842585 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:18.842585 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:18.842585 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:18.842585 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:18.842585 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:18.842585 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:18.842585 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:18.842585 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:55:18.842585 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:55:18.842585 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:55:18.842585 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:55:18.843762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:18.843762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:18.843762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:18.843762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:18.843762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:18.843762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:18.843762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:18.843762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:18.843762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:18.843762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:18.843762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:18.843762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:18.843762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:18.843762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:18.843762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:18.843762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:18.843762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:55:18.844676 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:55:18.844676 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:18.844676 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:18.844676 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 470/933] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:55:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:18.844676 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae081f5-4e33-4596-a668-d091df9da5cc None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:18.875336 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad5acf-d854-4f37-99f5-fc2e38fd06b6 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:18.917041 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc1def-b903-4e8b-b6d5-257b6e5f0157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:18.917655 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc1def-b903-4e8b-b6d5-257b6e5f0157 None admin] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:18.917937 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc1def-b903-4e8b-b6d5-257b6e5f0157 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:18.918215 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc1def-b903-4e8b-b6d5-257b6e5f0157 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:18.918914 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc1def-b903-4e8b-b6d5-257b6e5f0157 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:18.920856 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc1def-b903-4e8b-b6d5-257b6e5f0157 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:18.946979 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 464/934] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 14:55:18 2022] POST /identity/v3/endpoints => generated 346 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:55:18.947806 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc1def-b903-4e8b-b6d5-257b6e5f0157 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:18.949236 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae081f5-4e33-4596-a668-d091df9da5cc None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:19.013077 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b242ad4e-ff4d-47df-bd5d-5d5c4c14a12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:19.014498 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242ad4e-ff4d-47df-bd5d-5d5c4c14a12b None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:19.014894 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242ad4e-ff4d-47df-bd5d-5d5c4c14a12b None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:19.015131 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242ad4e-ff4d-47df-bd5d-5d5c4c14a12b None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:19.015800 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b242ad4e-ff4d-47df-bd5d-5d5c4c14a12b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:19.016106 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b242ad4e-ff4d-47df-bd5d-5d5c4c14a12b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:19.017578 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b242ad4e-ff4d-47df-bd5d-5d5c4c14a12b None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:19.027985 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 471/935] 10.222.0.33 () {56 vars in 1126 bytes} [Sun Sep 11 14:55:18 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:19.029665 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b242ad4e-ff4d-47df-bd5d-5d5c4c14a12b None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:19.038556 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:19.101603 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc1def-b903-4e8b-b6d5-257b6e5f0157 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:19.103230 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:19.150094 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5f1d5-dedb-4386-bc76-93e9a4b1fb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:19.150748 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5f1d5-dedb-4386-bc76-93e9a4b1fb8b None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:19.151063 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5f1d5-dedb-4386-bc76-93e9a4b1fb8b None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:19.151367 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5f1d5-dedb-4386-bc76-93e9a4b1fb8b None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:19.152512 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c5f1d5-dedb-4386-bc76-93e9a4b1fb8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:19.152930 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c5f1d5-dedb-4386-bc76-93e9a4b1fb8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:19.154836 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c5f1d5-dedb-4386-bc76-93e9a4b1fb8b None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:19.158597 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b242ad4e-ff4d-47df-bd5d-5d5c4c14a12b None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:19.170389 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 465/936] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:55:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:19.177132 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5f1d5-dedb-4386-bc76-93e9a4b1fb8b None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:19.204603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33445672-a58c-45a3-9e1a-2af2cb134acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:19.205768 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33445672-a58c-45a3-9e1a-2af2cb134acc None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:19.206195 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33445672-a58c-45a3-9e1a-2af2cb134acc None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:19.206775 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33445672-a58c-45a3-9e1a-2af2cb134acc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:19.207977 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33445672-a58c-45a3-9e1a-2af2cb134acc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:19.209897 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33445672-a58c-45a3-9e1a-2af2cb134acc None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:19.237607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 472/937] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 14:55:19 2022] POST /identity/v3/endpoints => generated 347 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:55:19.300775 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5f1d5-dedb-4386-bc76-93e9a4b1fb8b None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:19.317896 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:19.349188 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ceab6-9643-44c7-8265-ec16f63668d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:19.349835 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ceab6-9643-44c7-8265-ec16f63668d9 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:19.350111 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ceab6-9643-44c7-8265-ec16f63668d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:19.350382 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ceab6-9643-44c7-8265-ec16f63668d9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:19.358947 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ceab6-9643-44c7-8265-ec16f63668d9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:19.360345 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ceab6-9643-44c7-8265-ec16f63668d9 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:19.373543 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-f20ceab6-9643-44c7-8265-ec16f63668d9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 14:55:19.373543 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:19.373543 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:19.373543 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:19.373543 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:19.373543 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:19.373543 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:19.373543 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:19.373543 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:19.373543 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:19.373543 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:19.373543 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:19.373543 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:55:19.373543 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:55:19.373543 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:55:19.373543 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:55:19.374844 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:19.374844 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:19.374844 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:55:19.374844 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:55:19.374844 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:55:19.374844 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:55:19.374844 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:19.374844 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:19.374844 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:19.374844 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:19.374844 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:19.374844 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:19.374844 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:19.374844 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:19.374844 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:19.374844 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:19.374844 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:19.374844 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:19.375814 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:19.375814 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:19.375814 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:55:19.375814 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:55:19.375814 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:55:19.375814 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:55:19.375814 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:55:19.375814 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:55:19.375814 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 14:55:19.375814 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:19.375814 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 466/938] 10.222.0.33 () {56 vars in 1117 bytes} [Sun Sep 11 14:55:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:55:19.382903 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33445672-a58c-45a3-9e1a-2af2cb134acc None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:19.497581 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33445672-a58c-45a3-9e1a-2af2cb134acc None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:19.512251 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:19.537008 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79340102-28ff-4d40-951f-53c1b20a9f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:19.538060 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79340102-28ff-4d40-951f-53c1b20a9f20 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:19.538599 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79340102-28ff-4d40-951f-53c1b20a9f20 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:19.539095 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79340102-28ff-4d40-951f-53c1b20a9f20 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:19.540031 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79340102-28ff-4d40-951f-53c1b20a9f20 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:19.540539 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79340102-28ff-4d40-951f-53c1b20a9f20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:19.542195 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79340102-28ff-4d40-951f-53c1b20a9f20 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:19.562498 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 473/939] 10.222.0.33 () {56 vars in 1149 bytes} [Sun Sep 11 14:55:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:19.568583 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ceab6-9643-44c7-8265-ec16f63668d9 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:19.675258 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ceab6-9643-44c7-8265-ec16f63668d9 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:19.712873 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2b6529-978d-4c7f-adbf-88215d1ee1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:19.713505 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b6529-978d-4c7f-adbf-88215d1ee1fa None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:19.713786 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b6529-978d-4c7f-adbf-88215d1ee1fa None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:19.714057 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b6529-978d-4c7f-adbf-88215d1ee1fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:19.726073 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2b6529-978d-4c7f-adbf-88215d1ee1fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:19.727559 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2b6529-978d-4c7f-adbf-88215d1ee1fa None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:19.742042 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-3f2b6529-978d-4c7f-adbf-88215d1ee1fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:19.742042 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:19.742042 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:19.742042 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:19.742042 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:19.742042 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:19.742042 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:19.742042 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:19.742042 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:19.742042 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:19.742042 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:19.742042 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:19.742042 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:55:19.742042 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:55:19.742042 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:55:19.742042 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:55:19.743020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:19.743020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:19.743020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:19.743020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:19.743020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:19.743020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:19.743020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:19.743020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:19.743020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:19.743020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:19.743020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:19.743020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:19.743020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:19.743020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:19.743020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:19.743020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:19.743020 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:55:19.743923 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:55:19.743923 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:19.743923 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:19.743923 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 467/940] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:55:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:19.750790 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79340102-28ff-4d40-951f-53c1b20a9f20 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:19.860755 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79340102-28ff-4d40-951f-53c1b20a9f20 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:19.899178 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8726f4-be19-4081-95e3-91439e37a385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:19.900159 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8726f4-be19-4081-95e3-91439e37a385 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:19.900641 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8726f4-be19-4081-95e3-91439e37a385 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:19.901173 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8726f4-be19-4081-95e3-91439e37a385 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:19.902088 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8726f4-be19-4081-95e3-91439e37a385 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:19.902589 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8726f4-be19-4081-95e3-91439e37a385 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:19.904243 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8726f4-be19-4081-95e3-91439e37a385 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:19.916794 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 474/941] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:55:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:19.923504 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2b6529-978d-4c7f-adbf-88215d1ee1fa None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:19.953413 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:20.037905 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2b6529-978d-4c7f-adbf-88215d1ee1fa None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:20.075226 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9112685a-9ef0-4e45-a142-80bf3c9bfef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:20.076216 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9112685a-9ef0-4e45-a142-80bf3c9bfef7 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:20.076517 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9112685a-9ef0-4e45-a142-80bf3c9bfef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:20.076856 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9112685a-9ef0-4e45-a142-80bf3c9bfef7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:20.083847 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9112685a-9ef0-4e45-a142-80bf3c9bfef7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:20.085415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9112685a-9ef0-4e45-a142-80bf3c9bfef7 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:20.097002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-9112685a-9ef0-4e45-a142-80bf3c9bfef7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:55:20.097002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:20.097002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:20.097002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:20.097002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:20.097002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:20.097002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:20.097002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:20.097002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:20.097002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:20.097002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:20.097002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:20.097002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:55:20.097002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:55:20.097002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:55:20.097913 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:55:20.097913 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:20.097913 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:20.097913 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:20.097913 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:20.097913 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:20.097913 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:20.097913 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:20.097913 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:20.097913 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:20.097913 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:20.097913 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:20.097913 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:20.097913 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:20.097913 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:20.097913 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:20.097913 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:20.098840 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:55:20.098840 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:55:20.098840 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:55:20.098840 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:55:20.098840 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:55:20.098840 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:55:20.098840 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:55:20.098840 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:20.098840 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 468/942] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:55:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:20.105585 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8726f4-be19-4081-95e3-91439e37a385 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:20.139754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:20.213195 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8726f4-be19-4081-95e3-91439e37a385 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:20.250425 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfa71a4-5a0f-4193-aedb-b67a6398cc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:20.251449 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa71a4-5a0f-4193-aedb-b67a6398cc5e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:20.251934 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa71a4-5a0f-4193-aedb-b67a6398cc5e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:20.252426 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa71a4-5a0f-4193-aedb-b67a6398cc5e None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:20.253327 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfa71a4-5a0f-4193-aedb-b67a6398cc5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:20.253832 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfa71a4-5a0f-4193-aedb-b67a6398cc5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:20.255495 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfa71a4-5a0f-4193-aedb-b67a6398cc5e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:20.270068 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 475/943] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:55:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:20.276633 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9112685a-9ef0-4e45-a142-80bf3c9bfef7 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:20.363436 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:20.382372 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9112685a-9ef0-4e45-a142-80bf3c9bfef7 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:20.406397 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e1970-20c2-4dd3-9a39-1c684a954e50 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:20.407053 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e1970-20c2-4dd3-9a39-1c684a954e50 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:20.407553 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e1970-20c2-4dd3-9a39-1c684a954e50 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:20.413001 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 476/944] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:55:20.414842 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3d8d9-4446-4e2b-a39a-7d7922bd202a None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:20.415339 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3d8d9-4446-4e2b-a39a-7d7922bd202a None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:20.415807 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3d8d9-4446-4e2b-a39a-7d7922bd202a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:20.417247 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd1543-616f-4402-83b4-0019aff97671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:20.418011 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd1543-616f-4402-83b4-0019aff97671 None admin] REQUEST_METHOD: `PUT` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:20.418296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd1543-616f-4402-83b4-0019aff97671 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:20.418570 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd1543-616f-4402-83b4-0019aff97671 None admin] PATH_INFO: `/v3/projects/9e4a3d44808e4b4092cfc4ee284c0176/users/0aafb2bbba1d466fb22bb0824a531ddc/roles/56ed900824b74d938089a39aeb349cf1` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:20.421588 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd1543-616f-4402-83b4-0019aff97671 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e4a3d44808e4b4092cfc4ee284c0176, user_id=0aafb2bbba1d466fb22bb0824a531ddc, role_id=56ed900824b74d938089a39aeb349cf1)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:20.423041 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd1543-616f-4402-83b4-0019aff97671 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:20.439285 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-29f3d8d9-4446-4e2b-a39a-7d7922bd202a None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:20.441059 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-29f3d8d9-4446-4e2b-a39a-7d7922bd202a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:20.444629 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 469/945] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 14:55:20 2022] PUT /identity/v3/projects/9e4a3d44808e4b4092cfc4ee284c0176/users/0aafb2bbba1d466fb22bb0824a531ddc/roles/56ed900824b74d938089a39aeb349cf1 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:55:20.527012 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 477/946] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:20.803735 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd1543-616f-4402-83b4-0019aff97671 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:20.910799 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd1543-616f-4402-83b4-0019aff97671 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:20.948365 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c7dd68-fcdd-4552-8c1c-e9a93978af3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:20.949009 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7dd68-fcdd-4552-8c1c-e9a93978af3f None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:20.949291 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7dd68-fcdd-4552-8c1c-e9a93978af3f None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:20.949562 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7dd68-fcdd-4552-8c1c-e9a93978af3f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:20.950147 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c7dd68-fcdd-4552-8c1c-e9a93978af3f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:20.952248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c7dd68-fcdd-4552-8c1c-e9a93978af3f None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:20.961322 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-36c7dd68-fcdd-4552-8c1c-e9a93978af3f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 14:55:20.961322 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:20.961322 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:55:20.961322 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:55:20.961322 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:55:20.961322 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:55:20.961322 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:55:20.961322 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:20.961322 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 14:55:20.961322 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:20.961322 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:55:20.961322 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:20.961322 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:20.961322 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:20.961322 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:20.961322 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:20.961322 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:20.962320 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:20.962320 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:20.962320 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:20.962320 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:20.962320 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:20.962320 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:20.962320 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:55:20.962320 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:55:20.962320 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:55:20.962320 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:55:20.962320 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:20.962320 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:20.962320 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:20.962320 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:20.962320 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:20.962320 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:20.963580 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:20.963580 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:20.963580 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:20.963580 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:20.963580 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:20.963580 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:20.963580 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:20.963580 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:20.963580 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:20.963580 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:20.963580 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:55:20.963580 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:20.963580 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 14:55:20.963580 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:20.963580 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 470/947] 10.222.0.33 () {56 vars in 1126 bytes} [Sun Sep 11 14:55:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:20.981181 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f3d8d9-4446-4e2b-a39a-7d7922bd202a None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:21.012310 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:21.104484 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f3d8d9-4446-4e2b-a39a-7d7922bd202a None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:21.141344 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401ad67-08d5-4474-a0f4-a815c34837c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:21.142023 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401ad67-08d5-4474-a0f4-a815c34837c6 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:21.142308 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401ad67-08d5-4474-a0f4-a815c34837c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:21.142585 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401ad67-08d5-4474-a0f4-a815c34837c6 None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:21.143255 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b401ad67-08d5-4474-a0f4-a815c34837c6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:21.143560 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b401ad67-08d5-4474-a0f4-a815c34837c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:21.144946 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b401ad67-08d5-4474-a0f4-a815c34837c6 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:21.150164 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 478/948] 10.222.0.33 () {56 vars in 1122 bytes} [Sun Sep 11 14:55:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:21.156399 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c7dd68-fcdd-4552-8c1c-e9a93978af3f None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:21.166076 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:21.272969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c7dd68-fcdd-4552-8c1c-e9a93978af3f None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:21.312795 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef030b5-0c20-4b70-bc24-37bbe1e23d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:21.313759 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef030b5-0c20-4b70-bc24-37bbe1e23d69 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:21.314258 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef030b5-0c20-4b70-bc24-37bbe1e23d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:21.314726 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef030b5-0c20-4b70-bc24-37bbe1e23d69 None admin] PATH_INFO: `/v3/services` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:21.315602 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef030b5-0c20-4b70-bc24-37bbe1e23d69 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:21.316099 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef030b5-0c20-4b70-bc24-37bbe1e23d69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:21.317722 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef030b5-0c20-4b70-bc24-37bbe1e23d69 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:21.326937 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 471/949] 10.222.0.33 () {56 vars in 1122 bytes} [Sun Sep 11 14:55:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:21.330007 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401ad67-08d5-4474-a0f4-a815c34837c6 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:21.372057 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e90efd-5405-434e-818e-cf95cd53f087 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:21.372344 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e90efd-5405-434e-818e-cf95cd53f087 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:21.373708 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e90efd-5405-434e-818e-cf95cd53f087 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:21.376288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 472/950] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:55:21.384120 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158069a-0655-4d47-b5a9-b9700a45d945 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:21.384516 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158069a-0655-4d47-b5a9-b9700a45d945 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:21.384884 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158069a-0655-4d47-b5a9-b9700a45d945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:21.405197 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-9158069a-0655-4d47-b5a9-b9700a45d945 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:21.406542 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-9158069a-0655-4d47-b5a9-b9700a45d945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:21.428347 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:21.447269 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401ad67-08d5-4474-a0f4-a815c34837c6 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:21.490678 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:21.492680 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e5ec1-8900-4fd7-bf91-5ef933556ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:21.493774 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e5ec1-8900-4fd7-bf91-5ef933556ea5 None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:21.494155 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e5ec1-8900-4fd7-bf91-5ef933556ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:21.494528 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e5ec1-8900-4fd7-bf91-5ef933556ea5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:21.495352 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74e5ec1-8900-4fd7-bf91-5ef933556ea5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:21.497292 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74e5ec1-8900-4fd7-bf91-5ef933556ea5 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:21.506849 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 473/951] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 14:55:21.526287 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 479/952] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 14:55:21 2022] POST /identity/v3/endpoints => generated 349 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:55:21.592858 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6cbf18-2dd2-450d-9be3-7672e29cde78 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:21.593224 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6cbf18-2dd2-450d-9be3-7672e29cde78 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:21.593461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6cbf18-2dd2-450d-9be3-7672e29cde78 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:21.595147 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 474/953] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:21.600689 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251f0ab-a7cb-4aa2-a7b9-b5af3be25621 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:21.600977 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251f0ab-a7cb-4aa2-a7b9-b5af3be25621 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:21.601253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251f0ab-a7cb-4aa2-a7b9-b5af3be25621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:21.622448 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-f251f0ab-a7cb-4aa2-a7b9-b5af3be25621 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:21.623768 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-f251f0ab-a7cb-4aa2-a7b9-b5af3be25621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:21.711571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 480/954] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:21.790211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6cbf18-2dd2-450d-9be3-7672e29cde78 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:21.897704 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6cbf18-2dd2-450d-9be3-7672e29cde78 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:21.932382 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca5687-dde1-4517-9043-a5d92e4b7852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:21.933001 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca5687-dde1-4517-9043-a5d92e4b7852 None admin] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:21.933281 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca5687-dde1-4517-9043-a5d92e4b7852 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:21.933810 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca5687-dde1-4517-9043-a5d92e4b7852 None admin] PATH_INFO: `/v3/projects` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:21.934496 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca5687-dde1-4517-9043-a5d92e4b7852 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:21.936467 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca5687-dde1-4517-9043-a5d92e4b7852 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:21.939104 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: WARNING keystone.server.flask.common [None req-e0ca5687-dde1-4517-9043-a5d92e4b7852 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 11 14:55:21.958209 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.sql.core [None req-e0ca5687-dde1-4517-9043-a5d92e4b7852 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 11 14:55:21.958209 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 11 14:55:21.958209 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [parameters: {'id': 'd6de1755861b4bda8dc129787850af79', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 11 14:55:21.958209 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81431) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 11 14:55:21.975724 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-e0ca5687-dde1-4517-9043-a5d92e4b7852 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d6de1755861b4bda8dc129787850af79.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d6de1755861b4bda8dc129787850af79. Sep 11 14:55:21.975724 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:21.975724 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 14:55:21.975724 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 14:55:21.975724 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 14:55:21.975724 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 14:55:21.975724 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 14:55:21.975724 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 14:55:21.975724 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 14:55:21.975724 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application conn.query(q) Sep 11 14:55:21.975724 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 14:55:21.975724 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 14:55:21.975724 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 14:55:21.976560 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application result.read() Sep 11 14:55:21.976560 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 14:55:21.976560 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 14:55:21.976560 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 14:55:21.976560 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 14:55:21.976560 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 14:55:21.976560 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 14:55:21.976560 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 14:55:21.976560 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 14:55:21.976560 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 11 14:55:21.976560 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:21.976560 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 11 14:55:21.976560 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:21.976560 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:21.976560 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 11 14:55:21.976560 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 11 14:55:21.976560 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 11 14:55:21.976560 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 11 14:55:21.977941 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 14:55:21.977941 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application next(self.gen) Sep 11 14:55:21.977941 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 11 14:55:21.977941 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application yield resource Sep 11 14:55:21.977941 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 14:55:21.977941 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application next(self.gen) Sep 11 14:55:21.977941 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 11 14:55:21.977941 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application self.session.rollback() Sep 11 14:55:21.977941 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 11 14:55:21.977941 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application self.force_reraise() Sep 11 14:55:21.977941 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 11 14:55:21.977941 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise self.value Sep 11 14:55:21.977941 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 11 14:55:21.977941 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 11 14:55:21.977941 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 11 14:55:21.977941 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application session.commit() Sep 11 14:55:21.977941 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 11 14:55:21.979070 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 11 14:55:21.979070 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 11 14:55:21.979070 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application self._prepare_impl() Sep 11 14:55:21.979070 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 11 14:55:21.979070 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application self.session.flush() Sep 11 14:55:21.979070 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 11 14:55:21.979070 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application self._flush(objects) Sep 11 14:55:21.979070 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 11 14:55:21.979070 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 11 14:55:21.979070 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 11 14:55:21.979070 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application compat.raise_( Sep 11 14:55:21.979070 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 14:55:21.979070 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception Sep 11 14:55:21.979070 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 11 14:55:21.979070 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application flush_context.execute() Sep 11 14:55:21.979070 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 11 14:55:21.979070 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rec.execute(self) Sep 11 14:55:21.979969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 11 14:55:21.979969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 11 14:55:21.979969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 11 14:55:21.979969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 11 14:55:21.979969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 11 14:55:21.979969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 11 14:55:21.979969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 11 14:55:21.979969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 11 14:55:21.979969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 11 14:55:21.979969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 11 14:55:21.979969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 11 14:55:21.979969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 11 14:55:21.979969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 11 14:55:21.979969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 11 14:55:21.979969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 11 14:55:21.980740 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 11 14:55:21.980740 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 14:55:21.980740 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception Sep 11 14:55:21.980740 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 14:55:21.980740 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 14:55:21.980740 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 14:55:21.980740 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 14:55:21.980740 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 14:55:21.980740 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 14:55:21.980740 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 14:55:21.980740 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application conn.query(q) Sep 11 14:55:21.980740 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 14:55:21.980740 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 14:55:21.980740 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 14:55:21.980740 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application result.read() Sep 11 14:55:21.980740 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 14:55:21.980740 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 14:55:21.981597 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 14:55:21.981597 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 14:55:21.981597 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 14:55:21.981597 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 14:55:21.981597 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 14:55:21.981597 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 14:55:21.981597 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 11 14:55:21.981597 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 11 14:55:21.981597 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [parameters: {'id': 'd6de1755861b4bda8dc129787850af79', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 11 14:55:21.981597 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 11 14:55:21.981597 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:21.981597 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:55:21.981597 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:21.981597 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:21.981597 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 11 14:55:21.983759 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 11 14:55:21.983759 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 11 14:55:21.983759 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 11 14:55:21.983759 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 11 14:55:21.983759 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:21.983759 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:55:21.983759 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:21.983759 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:21.983759 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:21.983759 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:21.983759 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:21.983759 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:21.983759 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:21.983759 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:21.983759 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:21.983759 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:21.983759 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:21.984870 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:21.984870 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 11 14:55:21.984870 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 11 14:55:21.984870 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:21.984870 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:21.984870 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 11 14:55:21.984870 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 11 14:55:21.984870 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d6de1755861b4bda8dc129787850af79. Sep 11 14:55:21.984870 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:21.984870 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 475/955] 10.222.0.33 () {60 vars in 1140 bytes} [Sun Sep 11 14:55:21 2022] POST /identity/v3/projects => generated 262 bytes in 187 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:21.989402 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f251f0ab-a7cb-4aa2-a7b9-b5af3be25621 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:21.996076 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca5687-dde1-4517-9043-a5d92e4b7852 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:22.058988 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:22.082097 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:22.117971 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca5687-dde1-4517-9043-a5d92e4b7852 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:22.121299 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f251f0ab-a7cb-4aa2-a7b9-b5af3be25621 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:22.158724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29fa17-0f2d-4cad-a37c-171be5d0ce96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:22.158960 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546e97e1-95c4-4a9f-b3dd-6e2a00b8de9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:22.159304 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546e97e1-95c4-4a9f-b3dd-6e2a00b8de9b None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:22.159466 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29fa17-0f2d-4cad-a37c-171be5d0ce96 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:22.159607 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546e97e1-95c4-4a9f-b3dd-6e2a00b8de9b None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:22.159859 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546e97e1-95c4-4a9f-b3dd-6e2a00b8de9b None admin] PATH_INFO: `/v3/services/compute` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:22.159955 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29fa17-0f2d-4cad-a37c-171be5d0ce96 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:22.160275 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29fa17-0f2d-4cad-a37c-171be5d0ce96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:22.160442 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546e97e1-95c4-4a9f-b3dd-6e2a00b8de9b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:22.162152 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546e97e1-95c4-4a9f-b3dd-6e2a00b8de9b None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:22.167497 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29fa17-0f2d-4cad-a37c-171be5d0ce96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:22.169014 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29fa17-0f2d-4cad-a37c-171be5d0ce96 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:22.172494 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-546e97e1-95c4-4a9f-b3dd-6e2a00b8de9b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 14:55:22.172494 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:22.172494 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:55:22.172494 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:55:22.172494 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:55:22.172494 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:55:22.172494 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:55:22.172494 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:22.172494 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 14:55:22.172494 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:22.172494 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:55:22.172494 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:22.172494 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:22.172494 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:22.172494 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:22.172494 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:22.172494 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:22.173601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:22.173601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:22.173601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:22.173601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:22.173601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:22.173601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:22.173601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:55:22.173601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:55:22.173601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:55:22.173601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:55:22.173601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:22.173601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:22.173601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:22.173601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:22.173601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:22.173601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:22.174508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:22.174508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:22.174508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:22.174508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:22.174508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:22.174508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:22.174508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:22.174508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:22.174508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:22.174508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:22.174508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:55:22.174508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:22.174508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 14:55:22.174508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:22.174508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 476/956] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:55:21 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:22.184014 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546e97e1-95c4-4a9f-b3dd-6e2a00b8de9b None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:22.185548 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-0d29fa17-0f2d-4cad-a37c-171be5d0ce96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:55:22.185548 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:22.185548 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:22.185548 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:22.185548 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:22.185548 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:22.185548 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:22.185548 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:22.185548 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:22.185548 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:22.185548 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:22.185548 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:22.185548 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:55:22.185548 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:55:22.185548 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:55:22.186482 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:55:22.186482 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:22.186482 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:22.186482 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:22.186482 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:22.186482 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:22.186482 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:22.186482 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:22.186482 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:22.186482 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:22.186482 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:22.186482 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:22.186482 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:22.186482 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:22.186482 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:22.186482 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:22.186482 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:22.187416 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:55:22.187416 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:55:22.187416 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:55:22.187416 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:55:22.187416 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:55:22.187416 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:55:22.187416 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:55:22.187416 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:22.187416 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 481/957] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:55:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:22.197558 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29fa17-0f2d-4cad-a37c-171be5d0ce96 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:22.299790 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546e97e1-95c4-4a9f-b3dd-6e2a00b8de9b None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:22.323832 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:22.341855 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a29d20f-a3f1-4a11-b499-89a519ffac92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:22.343152 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29d20f-a3f1-4a11-b499-89a519ffac92 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:22.343647 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29d20f-a3f1-4a11-b499-89a519ffac92 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:22.344148 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29d20f-a3f1-4a11-b499-89a519ffac92 None admin] PATH_INFO: `/v3/services` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:22.344994 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a29d20f-a3f1-4a11-b499-89a519ffac92 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:22.345519 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a29d20f-a3f1-4a11-b499-89a519ffac92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:22.347169 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a29d20f-a3f1-4a11-b499-89a519ffac92 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:22.348003 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29fa17-0f2d-4cad-a37c-171be5d0ce96 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:22.362597 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 477/958] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 11 14:55:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:22.364596 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a29d20f-a3f1-4a11-b499-89a519ffac92 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:22.384998 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:22.397867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f4bc2-180e-450f-9432-601531d24b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:22.398479 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f4bc2-180e-450f-9432-601531d24b99 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:22.398756 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f4bc2-180e-450f-9432-601531d24b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:22.399157 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f4bc2-180e-450f-9432-601531d24b99 None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:22.400257 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f4bc2-180e-450f-9432-601531d24b99 None admin] RBAC: Adding query filter params (name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:22.400558 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f4bc2-180e-450f-9432-601531d24b99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:22.402694 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f4bc2-180e-450f-9432-601531d24b99 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:22.427953 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 482/959] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 11 14:55:22 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:22.492530 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a29d20f-a3f1-4a11-b499-89a519ffac92 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:22.534041 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47685b3e-567a-4062-b1c4-78792885b3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:22.535004 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47685b3e-567a-4062-b1c4-78792885b3f3 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:22.535452 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47685b3e-567a-4062-b1c4-78792885b3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:22.535875 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47685b3e-567a-4062-b1c4-78792885b3f3 None admin] PATH_INFO: `/v3/services` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:22.536677 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47685b3e-567a-4062-b1c4-78792885b3f3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:22.537149 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47685b3e-567a-4062-b1c4-78792885b3f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:22.538736 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47685b3e-567a-4062-b1c4-78792885b3f3 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:22.546605 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 478/960] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 11 14:55:22 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:55:22.823956 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d814d3-5ef5-4dcb-8aa4-0f9f8681f2a5 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:22.824348 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d814d3-5ef5-4dcb-8aa4-0f9f8681f2a5 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:22.824551 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d814d3-5ef5-4dcb-8aa4-0f9f8681f2a5 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:22.826245 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 483/961] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:22.831939 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6115b8f-dedd-4100-b185-5559fcdb6318 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:22.832222 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6115b8f-dedd-4100-b185-5559fcdb6318 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:22.832498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6115b8f-dedd-4100-b185-5559fcdb6318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:22.851890 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-f6115b8f-dedd-4100-b185-5559fcdb6318 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:22.853403 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-f6115b8f-dedd-4100-b185-5559fcdb6318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:22.902700 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:22.928936 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 479/962] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:23.201183 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d814d3-5ef5-4dcb-8aa4-0f9f8681f2a5 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:23.315418 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d814d3-5ef5-4dcb-8aa4-0f9f8681f2a5 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:23.352701 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6186af-fd6e-4a89-93fa-13f372f8f615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:23.353330 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6186af-fd6e-4a89-93fa-13f372f8f615 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:23.353609 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6186af-fd6e-4a89-93fa-13f372f8f615 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:23.353876 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6186af-fd6e-4a89-93fa-13f372f8f615 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:23.365022 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6186af-fd6e-4a89-93fa-13f372f8f615 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:55:23.369890 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-6f6186af-fd6e-4a89-93fa-13f372f8f615 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 14:55:23.369890 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:23.369890 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:23.369890 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:23.369890 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:23.369890 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:23.369890 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:23.369890 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:23.369890 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:23.369890 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:23.369890 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:23.369890 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:23.369890 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:55:23.369890 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:55:23.369890 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:55:23.369890 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:55:23.370849 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:55:23.370849 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:55:23.370849 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:55:23.370849 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:55:23.370849 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:23.370849 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:23.370849 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:23.370849 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:23.370849 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:23.370849 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:23.370849 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:23.370849 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:23.370849 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:23.370849 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:23.370849 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:23.370849 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:23.371724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:23.371724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:23.371724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:23.371724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:23.371724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:55:23.371724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:55:23.371724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:55:23.371724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:55:23.371724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:55:23.371724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:55:23.371724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 14:55:23.371724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:23.371724 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 484/963] 10.222.0.33 () {56 vars in 1120 bytes} [Sun Sep 11 14:55:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:23.377392 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6115b8f-dedd-4100-b185-5559fcdb6318 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:23.485041 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6115b8f-dedd-4100-b185-5559fcdb6318 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:23.521638 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a834b42f-e2ce-4119-9ba5-cfc52bfa4811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:23.522664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a834b42f-e2ce-4119-9ba5-cfc52bfa4811 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:23.523181 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a834b42f-e2ce-4119-9ba5-cfc52bfa4811 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:23.523706 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a834b42f-e2ce-4119-9ba5-cfc52bfa4811 None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:23.524625 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a834b42f-e2ce-4119-9ba5-cfc52bfa4811 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:23.525151 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a834b42f-e2ce-4119-9ba5-cfc52bfa4811 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:23.526785 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a834b42f-e2ce-4119-9ba5-cfc52bfa4811 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:23.537571 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 480/964] 10.222.0.33 () {56 vars in 1115 bytes} [Sun Sep 11 14:55:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:55:23.543035 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6186af-fd6e-4a89-93fa-13f372f8f615 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:23.594006 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:23.654441 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6186af-fd6e-4a89-93fa-13f372f8f615 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:23.691758 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8553a8-0c1c-44a7-9506-1ef012944f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:23.692635 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8553a8-0c1c-44a7-9506-1ef012944f3c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:23.693064 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8553a8-0c1c-44a7-9506-1ef012944f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:23.693527 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8553a8-0c1c-44a7-9506-1ef012944f3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:23.705012 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8553a8-0c1c-44a7-9506-1ef012944f3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:23.706674 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8553a8-0c1c-44a7-9506-1ef012944f3c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:23.722214 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-fc8553a8-0c1c-44a7-9506-1ef012944f3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:23.722214 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:23.722214 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:23.722214 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:23.722214 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:23.722214 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:23.722214 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:23.722214 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:23.722214 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:23.722214 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:23.722214 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:23.722214 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:23.722214 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:55:23.722214 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:55:23.722214 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:55:23.722214 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:55:23.723430 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:23.723430 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:23.723430 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:23.723430 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:23.723430 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:23.723430 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:23.723430 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:23.723430 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:23.723430 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:23.723430 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:23.723430 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:23.723430 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:23.723430 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:23.723430 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:23.723430 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:23.723430 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:23.723430 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:55:23.724550 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:55:23.724550 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:23.724550 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:23.726297 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 485/965] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:55:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:23.731708 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a834b42f-e2ce-4119-9ba5-cfc52bfa4811 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:23.777521 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:23.843353 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a834b42f-e2ce-4119-9ba5-cfc52bfa4811 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:23.883432 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a2da9-f9e9-499a-ab4e-8524fb683d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:23.884330 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a2da9-f9e9-499a-ab4e-8524fb683d43 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:23.884770 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a2da9-f9e9-499a-ab4e-8524fb683d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:23.885188 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a2da9-f9e9-499a-ab4e-8524fb683d43 None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:23.885973 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a2da9-f9e9-499a-ab4e-8524fb683d43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:23.886421 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a2da9-f9e9-499a-ab4e-8524fb683d43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:23.888034 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a2da9-f9e9-499a-ab4e-8524fb683d43 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:23.901857 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 481/966] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:55:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:23.908022 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8553a8-0c1c-44a7-9506-1ef012944f3c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:23.981506 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea276e8-e1e5-4b73-a8d3-056a50dbff24 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:23.983110 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea276e8-e1e5-4b73-a8d3-056a50dbff24 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:23.983681 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea276e8-e1e5-4b73-a8d3-056a50dbff24 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:23.993698 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 482/967] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:23 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:23.994657 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32754b2e-d474-4133-b6b8-cf564bb281fc None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:23.994997 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32754b2e-d474-4133-b6b8-cf564bb281fc None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:23.995268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32754b2e-d474-4133-b6b8-cf564bb281fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:24.016880 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:24.017351 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-32754b2e-d474-4133-b6b8-cf564bb281fc None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:24.019636 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-32754b2e-d474-4133-b6b8-cf564bb281fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:24.021203 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8553a8-0c1c-44a7-9506-1ef012944f3c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:24.056673 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14d865-9719-41ca-b7a5-bb3c32e8b3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:24.057234 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14d865-9719-41ca-b7a5-bb3c32e8b3f1 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:24.057470 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14d865-9719-41ca-b7a5-bb3c32e8b3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:24.057728 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14d865-9719-41ca-b7a5-bb3c32e8b3f1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:24.065931 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca14d865-9719-41ca-b7a5-bb3c32e8b3f1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:24.067486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca14d865-9719-41ca-b7a5-bb3c32e8b3f1 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:24.079342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-ca14d865-9719-41ca-b7a5-bb3c32e8b3f1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 14:55:24.079342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:24.079342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:24.079342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:24.079342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:24.079342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:24.079342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:24.079342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:24.079342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:24.079342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:24.079342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:24.079342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:24.079342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:55:24.079342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:55:24.079342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:55:24.079342 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:55:24.080318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:24.080318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:24.080318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:55:24.080318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:55:24.080318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:55:24.080318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:55:24.080318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:24.080318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:24.080318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:24.080318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:24.080318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:24.080318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:24.080318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:24.080318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:24.080318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:24.080318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:24.080318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:24.080318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:24.081485 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:24.081485 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:24.081485 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:55:24.081485 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:55:24.081485 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:55:24.081485 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:55:24.081485 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:55:24.081485 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:55:24.081485 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 14:55:24.081485 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:24.081485 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 486/968] 10.222.0.33 () {56 vars in 1117 bytes} [Sun Sep 11 14:55:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:24.086329 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14d865-9719-41ca-b7a5-bb3c32e8b3f1 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:24.110912 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 483/969] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:24.188968 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14d865-9719-41ca-b7a5-bb3c32e8b3f1 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:24.223154 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46de839-ba38-4c01-8490-232515b193ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:24.223779 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46de839-ba38-4c01-8490-232515b193ce None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:24.224069 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46de839-ba38-4c01-8490-232515b193ce None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:24.224348 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46de839-ba38-4c01-8490-232515b193ce None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:24.225067 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46de839-ba38-4c01-8490-232515b193ce None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:24.225366 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46de839-ba38-4c01-8490-232515b193ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:24.226799 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46de839-ba38-4c01-8490-232515b193ce None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:24.244555 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 487/970] 10.222.0.33 () {56 vars in 1149 bytes} [Sun Sep 11 14:55:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:24.250659 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32754b2e-d474-4133-b6b8-cf564bb281fc None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:24.282136 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:24.359870 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32754b2e-d474-4133-b6b8-cf564bb281fc None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:24.395973 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e645b-ce42-4779-acc1-76d3ce022824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:24.396929 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e645b-ce42-4779-acc1-76d3ce022824 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:24.397415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e645b-ce42-4779-acc1-76d3ce022824 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:24.397922 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e645b-ce42-4779-acc1-76d3ce022824 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:24.411092 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46de839-ba38-4c01-8490-232515b193ce None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:24.429970 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:24.473576 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43e645b-ce42-4779-acc1-76d3ce022824 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:24.475469 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43e645b-ce42-4779-acc1-76d3ce022824 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:24.507212 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:24.525114 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46de839-ba38-4c01-8490-232515b193ce None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:24.566125 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2997d79-ee19-498b-a24a-e8c1564d42d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:24.566779 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2997d79-ee19-498b-a24a-e8c1564d42d5 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:24.567140 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2997d79-ee19-498b-a24a-e8c1564d42d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:24.567465 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2997d79-ee19-498b-a24a-e8c1564d42d5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:24.568016 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2997d79-ee19-498b-a24a-e8c1564d42d5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:24.569504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2997d79-ee19-498b-a24a-e8c1564d42d5 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:24.580867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-d2997d79-ee19-498b-a24a-e8c1564d42d5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 14:55:24.580867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:24.580867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:55:24.580867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:55:24.580867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:55:24.580867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:55:24.580867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:55:24.580867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:24.580867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 14:55:24.580867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:24.580867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:55:24.580867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:24.580867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:24.580867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:24.580867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:24.580867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:24.580867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:24.581992 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:24.581992 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:24.581992 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:24.581992 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:24.581992 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:24.581992 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:24.581992 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:55:24.581992 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:55:24.581992 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:55:24.581992 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:55:24.581992 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:24.581992 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:24.581992 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:24.581992 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:24.581992 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:24.581992 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:24.582892 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:24.582892 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:24.582892 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:24.582892 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:24.582892 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:24.582892 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:24.582892 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:24.582892 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:24.582892 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:24.582892 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:24.582892 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:55:24.582892 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:24.582892 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 14:55:24.582892 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:24.587325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 488/971] 10.222.0.33 () {56 vars in 1126 bytes} [Sun Sep 11 14:55:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:24.588308 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2997d79-ee19-498b-a24a-e8c1564d42d5 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:24.595179 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 484/972] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:55:24 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 14:55:24.602323 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e645b-ce42-4779-acc1-76d3ce022824 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:24.684133 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:24.706467 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2997d79-ee19-498b-a24a-e8c1564d42d5 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:24.709785 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e645b-ce42-4779-acc1-76d3ce022824 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:24.740670 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:24.742839 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16619df9-56ad-4185-82e0-e1a15d6884e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:24.743910 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16619df9-56ad-4185-82e0-e1a15d6884e3 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:24.744156 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16619df9-56ad-4185-82e0-e1a15d6884e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:24.744418 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16619df9-56ad-4185-82e0-e1a15d6884e3 None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:24.745056 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16619df9-56ad-4185-82e0-e1a15d6884e3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:24.745357 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16619df9-56ad-4185-82e0-e1a15d6884e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:24.746745 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16619df9-56ad-4185-82e0-e1a15d6884e3 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:24.747946 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573356e-eee1-47f5-b6b7-bdc6c7100ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:24.748879 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573356e-eee1-47f5-b6b7-bdc6c7100ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:24.749162 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573356e-eee1-47f5-b6b7-bdc6c7100ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:24.749434 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573356e-eee1-47f5-b6b7-bdc6c7100ec6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:24.756099 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 489/973] 10.222.0.33 () {56 vars in 1122 bytes} [Sun Sep 11 14:55:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:24.756099 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85091cfd-2f58-44b0-b6cd-3e665fe647eb None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:24.756099 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85091cfd-2f58-44b0-b6cd-3e665fe647eb None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:24.756099 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85091cfd-2f58-44b0-b6cd-3e665fe647eb None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:24.756099 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 490/974] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:24.759555 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85091cfd-2f58-44b0-b6cd-3e665fe647eb None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:24.766217 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0573356e-eee1-47f5-b6b7-bdc6c7100ec6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:24.767718 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0573356e-eee1-47f5-b6b7-bdc6c7100ec6 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:24.783823 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-0573356e-eee1-47f5-b6b7-bdc6c7100ec6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:24.783823 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:24.783823 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:24.783823 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:24.783823 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:24.783823 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:24.783823 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:24.783823 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:24.783823 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:24.783823 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:24.783823 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:24.783823 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:24.783823 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:55:24.783823 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:55:24.783823 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:55:24.783823 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:55:24.784791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:24.784791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:24.784791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:24.784791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:24.784791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:24.784791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:24.784791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:24.784791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:24.784791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:24.784791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:24.784791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:24.784791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:24.784791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:24.784791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:24.784791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:24.784791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:24.784791 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:55:24.785685 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:55:24.785685 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:24.785685 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:24.785685 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 485/975] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:55:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:55:24.786455 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c7bef-c8ba-4c3d-b52f-ac4c73480215 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:24.786700 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c7bef-c8ba-4c3d-b52f-ac4c73480215 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:24.787009 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c7bef-c8ba-4c3d-b52f-ac4c73480215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:24.807180 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-ce6c7bef-c8ba-4c3d-b52f-ac4c73480215 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:24.808673 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-ce6c7bef-c8ba-4c3d-b52f-ac4c73480215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:24.877391 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85091cfd-2f58-44b0-b6cd-3e665fe647eb None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:24.890897 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 486/976] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:24.890897 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6c7bef-c8ba-4c3d-b52f-ac4c73480215 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:24.925543 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c402d9-c20a-44a7-95d3-2def7eae5e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:24.926411 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c402d9-c20a-44a7-95d3-2def7eae5e30 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:24.926700 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c402d9-c20a-44a7-95d3-2def7eae5e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:24.926998 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c402d9-c20a-44a7-95d3-2def7eae5e30 None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:24.927876 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c402d9-c20a-44a7-95d3-2def7eae5e30 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:24.928567 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c402d9-c20a-44a7-95d3-2def7eae5e30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:24.930211 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c402d9-c20a-44a7-95d3-2def7eae5e30 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:24.938932 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 491/977] 10.222.0.33 () {56 vars in 1122 bytes} [Sun Sep 11 14:55:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:24.939444 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09b813-fe96-4a86-a7ee-b5f3c6eadabd None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:24.939733 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09b813-fe96-4a86-a7ee-b5f3c6eadabd None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:24.940013 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09b813-fe96-4a86-a7ee-b5f3c6eadabd None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:24.941422 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 492/978] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:24.943661 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09b813-fe96-4a86-a7ee-b5f3c6eadabd None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:24.958878 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:25.011579 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6c7bef-c8ba-4c3d-b52f-ac4c73480215 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:25.026326 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:25.050100 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600dbedc-cd4b-42b1-9e42-1d118cea8fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:25.050717 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600dbedc-cd4b-42b1-9e42-1d118cea8fdd None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:25.051029 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600dbedc-cd4b-42b1-9e42-1d118cea8fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:25.051311 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600dbedc-cd4b-42b1-9e42-1d118cea8fdd None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:25.051945 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600dbedc-cd4b-42b1-9e42-1d118cea8fdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:25.052250 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600dbedc-cd4b-42b1-9e42-1d118cea8fdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:25.053633 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600dbedc-cd4b-42b1-9e42-1d118cea8fdd None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:25.066415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 487/979] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:55:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:25.067701 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae11490-3050-4443-814e-699fcbbebf2b None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:25.067994 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae11490-3050-4443-814e-699fcbbebf2b None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:25.068271 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae11490-3050-4443-814e-699fcbbebf2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:25.075115 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09b813-fe96-4a86-a7ee-b5f3c6eadabd None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:25.087164 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-aae11490-3050-4443-814e-699fcbbebf2b None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:25.088363 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-aae11490-3050-4443-814e-699fcbbebf2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:25.112409 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da82bb4f-0f6a-40d5-8b9c-14ad92c30084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:25.113315 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82bb4f-0f6a-40d5-8b9c-14ad92c30084 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:25.113783 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82bb4f-0f6a-40d5-8b9c-14ad92c30084 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:25.114053 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82bb4f-0f6a-40d5-8b9c-14ad92c30084 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:25.114757 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da82bb4f-0f6a-40d5-8b9c-14ad92c30084 None admin] RBAC: Adding query filter params (interface=admin, service_id=06804c3cbb45436d8341fe2633c2fe40, region_id=RegionOne) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:25.115107 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da82bb4f-0f6a-40d5-8b9c-14ad92c30084 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:25.116470 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da82bb4f-0f6a-40d5-8b9c-14ad92c30084 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:25.121643 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 493/980] 10.222.0.33 () {56 vars in 1249 bytes} [Sun Sep 11 14:55:24 2022] GET /identity/v3/endpoints?service_id=06804c3cbb45436d8341fe2633c2fe40&interface=admin®ion_id=RegionOne => generated 193 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:25.122386 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da82bb4f-0f6a-40d5-8b9c-14ad92c30084 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:25.165235 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 488/981] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:25.166084 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae11490-3050-4443-814e-699fcbbebf2b None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:25.209539 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:25.243575 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da82bb4f-0f6a-40d5-8b9c-14ad92c30084 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:25.272109 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae11490-3050-4443-814e-699fcbbebf2b None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:25.273638 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:25.281777 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4818a8d-0a76-4ac4-872c-336ed495e715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:25.282415 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4818a8d-0a76-4ac4-872c-336ed495e715 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:25.283073 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4818a8d-0a76-4ac4-872c-336ed495e715 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:25.283363 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4818a8d-0a76-4ac4-872c-336ed495e715 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:25.290119 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4818a8d-0a76-4ac4-872c-336ed495e715 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:25.291759 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4818a8d-0a76-4ac4-872c-336ed495e715 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:25.303617 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-a4818a8d-0a76-4ac4-872c-336ed495e715 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:55:25.303617 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:25.303617 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:25.303617 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:25.303617 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:25.303617 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:25.303617 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:25.303617 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:25.303617 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:25.303617 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:25.303617 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:25.303617 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:25.303617 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:55:25.303617 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:55:25.303617 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:55:25.304535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:55:25.304535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:25.304535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:25.304535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:25.304535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:25.304535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:25.304535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:25.304535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:25.304535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:25.304535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:25.304535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:25.304535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:25.304535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:25.304535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:25.304535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:25.304535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:25.304535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:25.305461 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:55:25.305461 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:55:25.305461 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:55:25.305461 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:55:25.305461 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:55:25.305461 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:55:25.305461 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:55:25.305461 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:25.305461 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 494/982] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:55:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:25.305461 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4818a8d-0a76-4ac4-872c-336ed495e715 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:25.314342 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e476dc-6453-4e3f-a0f5-2b2695a4a4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:25.314998 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e476dc-6453-4e3f-a0f5-2b2695a4a4c8 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:25.315287 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e476dc-6453-4e3f-a0f5-2b2695a4a4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:25.315564 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e476dc-6453-4e3f-a0f5-2b2695a4a4c8 None admin] PATH_INFO: `/v3/services` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:25.316180 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e476dc-6453-4e3f-a0f5-2b2695a4a4c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:25.317556 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e476dc-6453-4e3f-a0f5-2b2695a4a4c8 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:25.323707 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 489/983] 10.222.0.33 () {56 vars in 1087 bytes} [Sun Sep 11 14:55:25 2022] GET /identity/v3/services => generated 1497 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 14:55:25.324484 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e476dc-6453-4e3f-a0f5-2b2695a4a4c8 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:25.448272 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e476dc-6453-4e3f-a0f5-2b2695a4a4c8 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:25.451910 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4818a8d-0a76-4ac4-872c-336ed495e715 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:25.486683 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:25.491784 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954aa06-2311-4d6b-bff9-7313f9186a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:25.492515 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954aa06-2311-4d6b-bff9-7313f9186a9f None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:25.492732 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e249b7b-08e1-4b31-8020-226f4d3d99c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:25.492887 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954aa06-2311-4d6b-bff9-7313f9186a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:25.493075 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954aa06-2311-4d6b-bff9-7313f9186a9f None admin] PATH_INFO: `/v3/projects` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:25.493347 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e249b7b-08e1-4b31-8020-226f4d3d99c3 None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:25.493629 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e249b7b-08e1-4b31-8020-226f4d3d99c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:25.493741 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d954aa06-2311-4d6b-bff9-7313f9186a9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:25.493899 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e249b7b-08e1-4b31-8020-226f4d3d99c3 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:25.494062 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d954aa06-2311-4d6b-bff9-7313f9186a9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:25.494561 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e249b7b-08e1-4b31-8020-226f4d3d99c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:25.495533 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d954aa06-2311-4d6b-bff9-7313f9186a9f None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:25.496061 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e249b7b-08e1-4b31-8020-226f4d3d99c3 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:25.499775 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: WARNING keystone.server.flask.common [None req-2e249b7b-08e1-4b31-8020-226f4d3d99c3 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 11 14:55:25.513866 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 490/984] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:55:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:25.513866 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954aa06-2311-4d6b-bff9-7313f9186a9f None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:25.517705 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.sql.core [None req-2e249b7b-08e1-4b31-8020-226f4d3d99c3 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 11 14:55:25.517705 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 11 14:55:25.517705 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [parameters: {'user_id': 'a89fa810751d4a0689ebea52682c1031', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 11 14:55:25.517705 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81430) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 11 14:55:25.526752 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-2e249b7b-08e1-4b31-8020-226f4d3d99c3 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 11 14:55:25.526752 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:25.526752 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 14:55:25.526752 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 14:55:25.526752 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 14:55:25.526752 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 14:55:25.526752 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 14:55:25.526752 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 14:55:25.526752 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 14:55:25.526752 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application conn.query(q) Sep 11 14:55:25.526752 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 14:55:25.526752 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 14:55:25.526752 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 14:55:25.526752 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application result.read() Sep 11 14:55:25.526752 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 14:55:25.527689 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 14:55:25.527689 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 14:55:25.527689 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 14:55:25.527689 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 14:55:25.527689 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 14:55:25.527689 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 14:55:25.527689 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 14:55:25.527689 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 11 14:55:25.527689 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:25.527689 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 11 14:55:25.527689 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:25.527689 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:25.527689 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 11 14:55:25.527689 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 11 14:55:25.527689 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 11 14:55:25.527689 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 11 14:55:25.527689 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 14:55:25.527689 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application next(self.gen) Sep 11 14:55:25.529679 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 11 14:55:25.529679 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application yield resource Sep 11 14:55:25.529679 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 14:55:25.529679 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application next(self.gen) Sep 11 14:55:25.529679 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 11 14:55:25.529679 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self.session.rollback() Sep 11 14:55:25.529679 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 11 14:55:25.529679 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self.force_reraise() Sep 11 14:55:25.529679 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 11 14:55:25.529679 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise self.value Sep 11 14:55:25.529679 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 11 14:55:25.529679 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 11 14:55:25.529679 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 11 14:55:25.529679 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application session.commit() Sep 11 14:55:25.529679 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 11 14:55:25.529679 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 11 14:55:25.529679 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 11 14:55:25.531435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._prepare_impl() Sep 11 14:55:25.531435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 11 14:55:25.531435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self.session.flush() Sep 11 14:55:25.531435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 11 14:55:25.531435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._flush(objects) Sep 11 14:55:25.531435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 11 14:55:25.531435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 11 14:55:25.531435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 11 14:55:25.531435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application compat.raise_( Sep 11 14:55:25.531435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 14:55:25.531435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception Sep 11 14:55:25.531435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 11 14:55:25.531435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application flush_context.execute() Sep 11 14:55:25.531435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 11 14:55:25.531435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rec.execute(self) Sep 11 14:55:25.531435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 11 14:55:25.531435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 11 14:55:25.532356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 11 14:55:25.532356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 11 14:55:25.532356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 11 14:55:25.532356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 11 14:55:25.532356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 11 14:55:25.532356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 11 14:55:25.532356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 11 14:55:25.532356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 11 14:55:25.532356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 11 14:55:25.532356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 11 14:55:25.532356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 11 14:55:25.532356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 11 14:55:25.532356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 11 14:55:25.532356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 11 14:55:25.532356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 14:55:25.532356 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception Sep 11 14:55:25.536385 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 14:55:25.536385 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 14:55:25.536385 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 14:55:25.536385 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 14:55:25.536385 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 14:55:25.536385 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 14:55:25.536385 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 14:55:25.536385 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application conn.query(q) Sep 11 14:55:25.536385 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 14:55:25.536385 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 14:55:25.536385 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 14:55:25.536385 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application result.read() Sep 11 14:55:25.536385 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 14:55:25.536385 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 14:55:25.536385 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 14:55:25.536385 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 14:55:25.537603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 14:55:25.537603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 14:55:25.537603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 14:55:25.537603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 14:55:25.537603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 11 14:55:25.537603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 11 14:55:25.537603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [parameters: {'user_id': 'a89fa810751d4a0689ebea52682c1031', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 11 14:55:25.537603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 11 14:55:25.537603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:25.537603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:55:25.537603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:25.537603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:25.537603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:25.537603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:25.537603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:25.538851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:25.538851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:25.538851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:25.538851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:25.538851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:25.538851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:25.538851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:25.538851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 11 14:55:25.538851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 11 14:55:25.538851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:25.538851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:25.538851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:55:25.538851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:55:25.538851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:55:25.538851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:55:25.538851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 11 14:55:25.538851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 11 14:55:25.539918 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 11 14:55:25.539918 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 11 14:55:25.539918 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 11 14:55:25.539918 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 11 14:55:25.539918 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 11 14:55:25.539918 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:25.539918 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 495/985] 10.222.0.33 () {60 vars in 1131 bytes} [Sun Sep 11 14:55:25 2022] POST /identity/v3/users => generated 160 bytes in 223 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:25.539918 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e249b7b-08e1-4b31-8020-226f4d3d99c3 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:25.550146 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:25.626743 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954aa06-2311-4d6b-bff9-7313f9186a9f None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:25.652183 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e249b7b-08e1-4b31-8020-226f4d3d99c3 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:25.664534 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697e38c-a9d9-47b6-bcf8-c6480f3f27a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:25.665185 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697e38c-a9d9-47b6-bcf8-c6480f3f27a5 None admin] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:25.665485 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697e38c-a9d9-47b6-bcf8-c6480f3f27a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:25.665741 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697e38c-a9d9-47b6-bcf8-c6480f3f27a5 None admin] PATH_INFO: `/v3/services` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:25.666402 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6697e38c-a9d9-47b6-bcf8-c6480f3f27a5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:25.667814 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6697e38c-a9d9-47b6-bcf8-c6480f3f27a5 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:25.684024 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 491/986] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 11 14:55:25 2022] POST /identity/v3/services => generated 242 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:55:25.684659 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697e38c-a9d9-47b6-bcf8-c6480f3f27a5 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:25.714858 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb8b18-852a-4137-9622-79b6a2b68391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:25.716102 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb8b18-852a-4137-9622-79b6a2b68391 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:25.716630 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb8b18-852a-4137-9622-79b6a2b68391 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:25.717183 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb8b18-852a-4137-9622-79b6a2b68391 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:25.718150 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfb8b18-852a-4137-9622-79b6a2b68391 None admin] RBAC: Adding query filter params (role.id=56ed900824b74d938089a39aeb349cf1, scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176, user.id=0aafb2bbba1d466fb22bb0824a531ddc) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:25.718716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfb8b18-852a-4137-9622-79b6a2b68391 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:25.720872 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfb8b18-852a-4137-9622-79b6a2b68391 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:25.728815 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 496/987] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:55:25 2022] GET /identity/v3/role_assignments?user.id=0aafb2bbba1d466fb22bb0824a531ddc&scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176&role.id=56ed900824b74d938089a39aeb349cf1 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:25.794636 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:25.814271 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697e38c-a9d9-47b6-bcf8-c6480f3f27a5 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:25.855410 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f60c6-2280-494a-b056-55f6976197b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:25.856077 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f60c6-2280-494a-b056-55f6976197b9 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:25.856359 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f60c6-2280-494a-b056-55f6976197b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:25.856635 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f60c6-2280-494a-b056-55f6976197b9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:25.865990 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f60c6-2280-494a-b056-55f6976197b9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:25.867540 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f60c6-2280-494a-b056-55f6976197b9 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:25.881874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-935f60c6-2280-494a-b056-55f6976197b9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 14:55:25.881874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:25.881874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:25.881874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:25.881874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:25.881874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:25.881874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:25.881874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:25.881874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:25.881874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:25.881874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:25.881874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:25.881874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:55:25.881874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:55:25.881874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:55:25.881874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:55:25.882950 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:25.882950 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:25.882950 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:55:25.882950 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:55:25.882950 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:55:25.882950 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:55:25.882950 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:25.882950 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:25.882950 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:25.882950 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:25.882950 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:25.882950 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:25.882950 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:25.882950 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:25.882950 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:25.882950 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:25.882950 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:25.882950 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:25.884674 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:25.884674 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:25.884674 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:55:25.884674 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:55:25.884674 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:55:25.884674 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:55:25.884674 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:55:25.884674 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:55:25.884674 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 14:55:25.884674 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:25.884674 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 492/988] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:55:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:25.890183 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb8b18-852a-4137-9622-79b6a2b68391 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:25.954774 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:26.003188 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb8b18-852a-4137-9622-79b6a2b68391 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:26.040577 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df463a5-05ce-4696-8630-67005555c6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:26.041310 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df463a5-05ce-4696-8630-67005555c6a9 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:26.041598 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df463a5-05ce-4696-8630-67005555c6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:26.041873 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df463a5-05ce-4696-8630-67005555c6a9 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:26.042640 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df463a5-05ce-4696-8630-67005555c6a9 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:26.042973 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df463a5-05ce-4696-8630-67005555c6a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:26.044454 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df463a5-05ce-4696-8630-67005555c6a9 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:26.081371 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 497/989] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 14:55:25 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:55:27.613064 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84a7eb-9789-449d-b800-51fc10c4457c None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:27.613592 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84a7eb-9789-449d-b800-51fc10c4457c None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:27.613592 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84a7eb-9789-449d-b800-51fc10c4457c None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:27.615117 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 493/990] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:27.620190 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187a4f5-968d-4301-8cfc-89f895a825e6 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:27.620570 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187a4f5-968d-4301-8cfc-89f895a825e6 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:27.620747 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187a4f5-968d-4301-8cfc-89f895a825e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:27.639901 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-f187a4f5-968d-4301-8cfc-89f895a825e6 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:27.641396 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-f187a4f5-968d-4301-8cfc-89f895a825e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:27.711371 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 498/991] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:27 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:27.976274 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84a7eb-9789-449d-b800-51fc10c4457c None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:28.082609 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84a7eb-9789-449d-b800-51fc10c4457c None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:28.117352 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47858d4-9457-423e-8447-7aed24f0c1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:28.117987 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47858d4-9457-423e-8447-7aed24f0c1a4 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:28.118267 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47858d4-9457-423e-8447-7aed24f0c1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:28.118544 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47858d4-9457-423e-8447-7aed24f0c1a4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:28.119152 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47858d4-9457-423e-8447-7aed24f0c1a4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:28.120687 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47858d4-9457-423e-8447-7aed24f0c1a4 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:28.129642 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-d47858d4-9457-423e-8447-7aed24f0c1a4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 14:55:28.129642 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:28.129642 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:55:28.129642 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:55:28.129642 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:55:28.129642 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:55:28.129642 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:55:28.129642 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:28.129642 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 14:55:28.129642 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:28.129642 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:55:28.129642 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:28.129642 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:28.129642 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:28.129642 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:28.129642 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:28.129642 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:28.130621 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:28.130621 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:28.130621 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:28.130621 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:28.130621 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:28.130621 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:28.130621 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:55:28.130621 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:55:28.130621 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:55:28.130621 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:55:28.130621 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:28.130621 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:28.130621 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:28.130621 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:28.130621 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:28.130621 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:28.135976 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:28.135976 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:28.135976 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:28.135976 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:28.135976 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:28.135976 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:28.135976 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:28.135976 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:28.135976 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:28.135976 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:28.135976 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:55:28.135976 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:28.135976 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 14:55:28.135976 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:28.135976 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 494/992] 10.222.0.33 () {56 vars in 1126 bytes} [Sun Sep 11 14:55:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:28.136867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f187a4f5-968d-4301-8cfc-89f895a825e6 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:28.181542 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:28.240701 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f187a4f5-968d-4301-8cfc-89f895a825e6 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:28.276637 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed0f53d-fb1a-4e07-9241-6545af21b986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:28.277237 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0f53d-fb1a-4e07-9241-6545af21b986 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:28.277510 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0f53d-fb1a-4e07-9241-6545af21b986 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:28.277774 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0f53d-fb1a-4e07-9241-6545af21b986 None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:28.278391 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed0f53d-fb1a-4e07-9241-6545af21b986 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:28.278703 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed0f53d-fb1a-4e07-9241-6545af21b986 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:28.280256 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed0f53d-fb1a-4e07-9241-6545af21b986 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:28.285279 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 499/993] 10.222.0.33 () {56 vars in 1122 bytes} [Sun Sep 11 14:55:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:55:28.290718 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47858d4-9457-423e-8447-7aed24f0c1a4 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:28.304341 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:28.393283 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47858d4-9457-423e-8447-7aed24f0c1a4 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:28.427789 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde95ed2-4340-40de-a8e9-dda04bf2119e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:28.428394 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde95ed2-4340-40de-a8e9-dda04bf2119e None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:28.428670 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde95ed2-4340-40de-a8e9-dda04bf2119e None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:28.428938 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde95ed2-4340-40de-a8e9-dda04bf2119e None admin] PATH_INFO: `/v3/services` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:28.429562 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde95ed2-4340-40de-a8e9-dda04bf2119e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:28.429860 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde95ed2-4340-40de-a8e9-dda04bf2119e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:28.431262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde95ed2-4340-40de-a8e9-dda04bf2119e None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:28.437229 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 495/994] 10.222.0.33 () {56 vars in 1122 bytes} [Sun Sep 11 14:55:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:28.442639 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed0f53d-fb1a-4e07-9241-6545af21b986 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:28.541424 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed0f53d-fb1a-4e07-9241-6545af21b986 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:28.549288 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:28.577470 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdb031d-cb61-4c33-b1de-756c0641b7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:28.578078 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb031d-cb61-4c33-b1de-756c0641b7f4 None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:28.578352 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb031d-cb61-4c33-b1de-756c0641b7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:28.578619 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb031d-cb61-4c33-b1de-756c0641b7f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:28.579312 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdb031d-cb61-4c33-b1de-756c0641b7f4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:28.580685 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdb031d-cb61-4c33-b1de-756c0641b7f4 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:28.602832 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 500/995] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 14:55:28 2022] POST /identity/v3/endpoints => generated 325 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:55:28.879495 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9ccfb-d881-40a8-8e60-edddeb907222 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:28.880009 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9ccfb-d881-40a8-8e60-edddeb907222 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:28.880357 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9ccfb-d881-40a8-8e60-edddeb907222 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:28.883170 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 496/996] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:28.888293 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818eeef-bbff-48d5-89b3-ea3114ec2fbb None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:28.888982 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818eeef-bbff-48d5-89b3-ea3114ec2fbb None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:28.888982 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818eeef-bbff-48d5-89b3-ea3114ec2fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:28.907512 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-1818eeef-bbff-48d5-89b3-ea3114ec2fbb None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:28.908708 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-1818eeef-bbff-48d5-89b3-ea3114ec2fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:28.921341 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52c532-b115-47b8-8b33-1eba5cbf2a4f None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:28.921617 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52c532-b115-47b8-8b33-1eba5cbf2a4f None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:28.921889 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52c532-b115-47b8-8b33-1eba5cbf2a4f None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:28.923290 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 497/997] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:55:28.928358 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c04450-148c-4554-8e4c-751d23bce899 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:28.928633 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c04450-148c-4554-8e4c-751d23bce899 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:28.928900 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c04450-148c-4554-8e4c-751d23bce899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:28.949104 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-e6c04450-148c-4554-8e4c-751d23bce899 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:28.950296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-e6c04450-148c-4554-8e4c-751d23bce899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:28.999515 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 501/998] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:28 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:29.026857 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 498/999] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:28 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 14:55:29.264657 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1818eeef-bbff-48d5-89b3-ea3114ec2fbb None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:29.290161 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c04450-148c-4554-8e4c-751d23bce899 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:29.293273 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:29.361381 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:29.372317 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1818eeef-bbff-48d5-89b3-ea3114ec2fbb None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:29.405217 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c04450-148c-4554-8e4c-751d23bce899 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:29.410073 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690cb729-d96f-48d2-ab77-ac3ac0b22442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:29.410709 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690cb729-d96f-48d2-ab77-ac3ac0b22442 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:29.411016 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690cb729-d96f-48d2-ab77-ac3ac0b22442 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:29.411286 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690cb729-d96f-48d2-ab77-ac3ac0b22442 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:29.411862 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690cb729-d96f-48d2-ab77-ac3ac0b22442 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:29.413233 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690cb729-d96f-48d2-ab77-ac3ac0b22442 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:29.422158 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-690cb729-d96f-48d2-ab77-ac3ac0b22442 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 14:55:29.422158 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:29.422158 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:55:29.422158 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:55:29.422158 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:55:29.422158 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:55:29.422158 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:55:29.422158 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:29.422158 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 14:55:29.422158 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:29.422158 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:55:29.422158 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:29.422158 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:29.422158 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:29.422158 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:29.422158 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:29.422158 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:29.423238 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:29.423238 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:29.423238 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:29.423238 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:29.423238 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:29.423238 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:29.423238 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:55:29.423238 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:55:29.423238 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:55:29.423238 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:55:29.423238 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:29.423238 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:29.423238 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:29.423238 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:29.423238 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:29.423238 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:29.424104 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:29.424104 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:29.424104 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:29.424104 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:29.424104 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:29.424104 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:29.424104 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:29.424104 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:29.424104 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:29.424104 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:29.424104 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:55:29.424104 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:29.424104 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 14:55:29.424104 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:29.424104 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 502/1000] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:55:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:29.428911 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690cb729-d96f-48d2-ab77-ac3ac0b22442 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:29.446795 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b40ab44-ad34-4947-b985-0ccdf7638a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:29.447564 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40ab44-ad34-4947-b985-0ccdf7638a06 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:29.447914 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40ab44-ad34-4947-b985-0ccdf7638a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:29.448411 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40ab44-ad34-4947-b985-0ccdf7638a06 None admin] PATH_INFO: `/v3/services/image` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:29.449348 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b40ab44-ad34-4947-b985-0ccdf7638a06 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:29.451531 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b40ab44-ad34-4947-b985-0ccdf7638a06 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:29.460949 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-2b40ab44-ad34-4947-b985-0ccdf7638a06 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 14:55:29.460949 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:29.460949 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:55:29.460949 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:55:29.460949 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:55:29.460949 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:55:29.460949 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:55:29.460949 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:29.460949 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 14:55:29.460949 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:29.460949 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:55:29.460949 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:29.460949 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:29.460949 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:29.460949 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:29.460949 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:29.460949 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:29.461921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:29.461921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:29.461921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:29.461921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:29.461921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:29.461921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:29.461921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:55:29.461921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:55:29.461921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:55:29.461921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:55:29.461921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:29.461921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:29.461921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:29.461921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:29.461921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:29.461921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:29.462771 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:29.462771 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:29.462771 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:29.462771 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:29.462771 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:29.462771 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:29.462771 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:29.462771 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:29.462771 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:29.462771 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:29.462771 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:55:29.462771 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:29.462771 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 14:55:29.462771 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:29.462771 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 499/1001] 10.222.0.33 () {56 vars in 1105 bytes} [Sun Sep 11 14:55:29 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:55:29.467775 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b40ab44-ad34-4947-b985-0ccdf7638a06 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:29.539601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:29.542388 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690cb729-d96f-48d2-ab77-ac3ac0b22442 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:29.578098 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55135ad-d902-4cbc-bb36-0487b494af5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:29.579041 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55135ad-d902-4cbc-bb36-0487b494af5c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:29.579373 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55135ad-d902-4cbc-bb36-0487b494af5c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:29.579643 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55135ad-d902-4cbc-bb36-0487b494af5c None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:29.580264 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55135ad-d902-4cbc-bb36-0487b494af5c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:29.580559 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55135ad-d902-4cbc-bb36-0487b494af5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:29.580761 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b40ab44-ad34-4947-b985-0ccdf7638a06 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:29.581932 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55135ad-d902-4cbc-bb36-0487b494af5c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:29.591701 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 503/1002] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 11 14:55:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:55:29.592741 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55135ad-d902-4cbc-bb36-0487b494af5c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:29.615262 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:29.616232 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d76d28a-2f4f-45e0-9b72-2f679b093a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:29.616821 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d76d28a-2f4f-45e0-9b72-2f679b093a21 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:29.617093 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d76d28a-2f4f-45e0-9b72-2f679b093a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:29.617457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d76d28a-2f4f-45e0-9b72-2f679b093a21 None admin] PATH_INFO: `/v3/services` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:29.618177 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d76d28a-2f4f-45e0-9b72-2f679b093a21 None admin] RBAC: Adding query filter params (name=image) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:29.618479 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d76d28a-2f4f-45e0-9b72-2f679b093a21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:29.619904 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d76d28a-2f4f-45e0-9b72-2f679b093a21 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:29.629942 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 500/1003] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:55:29 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:55:29.630943 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d76d28a-2f4f-45e0-9b72-2f679b093a21 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:29.702454 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55135ad-d902-4cbc-bb36-0487b494af5c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:29.735894 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496dce6-b3b9-41f4-9ffc-2a7682757ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:29.736255 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d76d28a-2f4f-45e0-9b72-2f679b093a21 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:29.736503 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496dce6-b3b9-41f4-9ffc-2a7682757ede None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:29.736780 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496dce6-b3b9-41f4-9ffc-2a7682757ede None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:29.737047 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496dce6-b3b9-41f4-9ffc-2a7682757ede None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:29.737671 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1496dce6-b3b9-41f4-9ffc-2a7682757ede None admin] RBAC: Adding query filter params (type=compute) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:29.737969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1496dce6-b3b9-41f4-9ffc-2a7682757ede None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:29.739377 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1496dce6-b3b9-41f4-9ffc-2a7682757ede None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:29.744375 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 504/1004] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 11 14:55:29 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:29.751010 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496dce6-b3b9-41f4-9ffc-2a7682757ede None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:29.767713 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:29.773653 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950fed4-f988-4144-a946-37e4adc2c18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:29.774270 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950fed4-f988-4144-a946-37e4adc2c18c None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:29.774574 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950fed4-f988-4144-a946-37e4adc2c18c None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:29.774836 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950fed4-f988-4144-a946-37e4adc2c18c None admin] PATH_INFO: `/v3/services` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:29.775473 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7950fed4-f988-4144-a946-37e4adc2c18c None admin] RBAC: Adding query filter params (type=image) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:29.775775 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7950fed4-f988-4144-a946-37e4adc2c18c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:29.777667 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7950fed4-f988-4144-a946-37e4adc2c18c None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:29.790926 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 501/1005] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:55:29 2022] GET /identity/v3/services?type=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:55:29.855892 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:29.877727 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496dce6-b3b9-41f4-9ffc-2a7682757ede None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:29.915124 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfd4c9c-11f4-4816-9634-ace8fe4727d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:29.915840 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd4c9c-11f4-4816-9634-ace8fe4727d2 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:29.916102 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd4c9c-11f4-4816-9634-ace8fe4727d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:29.916288 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd4c9c-11f4-4816-9634-ace8fe4727d2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:29.916996 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfd4c9c-11f4-4816-9634-ace8fe4727d2 None admin] RBAC: Adding query filter params (interface=public, service_id=4da1ae9ab029400aad05cb8315bcd155, region_id=RegionOne) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:29.917296 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfd4c9c-11f4-4816-9634-ace8fe4727d2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:29.918685 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfd4c9c-11f4-4816-9634-ace8fe4727d2 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:29.930302 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 505/1006] 10.222.0.33 () {56 vars in 1251 bytes} [Sun Sep 11 14:55:29 2022] GET /identity/v3/endpoints?service_id=4da1ae9ab029400aad05cb8315bcd155&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:55:29.931553 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950fed4-f988-4144-a946-37e4adc2c18c None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:30.045539 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950fed4-f988-4144-a946-37e4adc2c18c None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:30.081158 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729bd097-056a-4e87-8b29-c9aa70b9b2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:30.081766 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729bd097-056a-4e87-8b29-c9aa70b9b2f6 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:30.082043 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729bd097-056a-4e87-8b29-c9aa70b9b2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:30.082318 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729bd097-056a-4e87-8b29-c9aa70b9b2f6 None admin] PATH_INFO: `/v3/services` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:30.082953 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729bd097-056a-4e87-8b29-c9aa70b9b2f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:30.084322 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729bd097-056a-4e87-8b29-c9aa70b9b2f6 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:30.098849 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 502/1007] 10.222.0.33 () {56 vars in 1087 bytes} [Sun Sep 11 14:55:29 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 14:55:30.815448 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472b712-aa53-44c1-bba4-ccd768c76441 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:30.815835 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472b712-aa53-44c1-bba4-ccd768c76441 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:30.816000 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472b712-aa53-44c1-bba4-ccd768c76441 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:30.817508 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 506/1008] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:30.822702 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b491a-ffdd-4828-88f1-e1adaa242401 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:30.823019 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b491a-ffdd-4828-88f1-e1adaa242401 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:30.823294 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b491a-ffdd-4828-88f1-e1adaa242401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:30.842162 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-203b491a-ffdd-4828-88f1-e1adaa242401 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:30.843496 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-203b491a-ffdd-4828-88f1-e1adaa242401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:30.899203 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:30.912221 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 503/1009] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:30 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:31.182174 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2472b712-aa53-44c1-bba4-ccd768c76441 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:31.287807 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2472b712-aa53-44c1-bba4-ccd768c76441 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:31.321866 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae598cc2-f1fb-4b66-a72e-05dbe8e7bd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:31.322495 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae598cc2-f1fb-4b66-a72e-05dbe8e7bd5f None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:31.322771 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae598cc2-f1fb-4b66-a72e-05dbe8e7bd5f None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:31.323065 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae598cc2-f1fb-4b66-a72e-05dbe8e7bd5f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:31.324176 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae598cc2-f1fb-4b66-a72e-05dbe8e7bd5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:31.326136 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae598cc2-f1fb-4b66-a72e-05dbe8e7bd5f None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:31.328270 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 507/1010] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:55:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:31.333886 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b491a-ffdd-4828-88f1-e1adaa242401 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:31.435840 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b491a-ffdd-4828-88f1-e1adaa242401 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:31.470298 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6a326-fee6-4d67-9323-505ba71aaf57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:31.470937 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6a326-fee6-4d67-9323-505ba71aaf57 None admin] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:31.471215 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6a326-fee6-4d67-9323-505ba71aaf57 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:31.471491 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6a326-fee6-4d67-9323-505ba71aaf57 None admin] PATH_INFO: `/v3/projects` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:31.472138 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e6a326-fee6-4d67-9323-505ba71aaf57 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:31.473567 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e6a326-fee6-4d67-9323-505ba71aaf57 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:31.506843 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 504/1011] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 11 14:55:31 2022] POST /identity/v3/projects => generated 311 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:55:31.987955 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9ced4-8b9d-4728-8f30-aa5daf2242dd None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:31.988353 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9ced4-8b9d-4728-8f30-aa5daf2242dd None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:31.988581 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9ced4-8b9d-4728-8f30-aa5daf2242dd None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:31.990261 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 508/1012] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:31.996252 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c76fa-19d8-48ec-a179-92c8c83b1b64 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:31.996730 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c76fa-19d8-48ec-a179-92c8c83b1b64 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:31.997179 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c76fa-19d8-48ec-a179-92c8c83b1b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:32.016012 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-ac4c76fa-19d8-48ec-a179-92c8c83b1b64 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:32.017196 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-ac4c76fa-19d8-48ec-a179-92c8c83b1b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:32.054792 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:32.090855 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 505/1013] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:31 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:32.252695 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51e4a3-3001-48a2-a7b1-f3f9a48d1fa0 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:32.253197 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51e4a3-3001-48a2-a7b1-f3f9a48d1fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:32.253197 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51e4a3-3001-48a2-a7b1-f3f9a48d1fa0 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:32.254627 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 509/1014] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:32.259669 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2ea94-327c-4636-96d2-2e10b8ff1350 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:32.259958 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2ea94-327c-4636-96d2-2e10b8ff1350 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:32.260236 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2ea94-327c-4636-96d2-2e10b8ff1350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:32.278904 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-0cf2ea94-327c-4636-96d2-2e10b8ff1350 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:32.280102 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-0cf2ea94-327c-4636-96d2-2e10b8ff1350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:32.349789 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 506/1015] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:32 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:32.367249 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51e4a3-3001-48a2-a7b1-f3f9a48d1fa0 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:32.386041 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:32.475444 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51e4a3-3001-48a2-a7b1-f3f9a48d1fa0 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:32.509105 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef7316-f2c9-4924-989e-061c0d46fb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:32.509646 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef7316-f2c9-4924-989e-061c0d46fb27 None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:32.509924 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef7316-f2c9-4924-989e-061c0d46fb27 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:32.510191 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef7316-f2c9-4924-989e-061c0d46fb27 None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:32.510857 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef7316-f2c9-4924-989e-061c0d46fb27 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:32.512223 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef7316-f2c9-4924-989e-061c0d46fb27 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:32.529200 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 510/1016] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 11 14:55:32 2022] POST /identity/v3/services => generated 242 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:55:32.620335 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf2ea94-327c-4636-96d2-2e10b8ff1350 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:32.729102 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf2ea94-327c-4636-96d2-2e10b8ff1350 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:32.730678 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:32.772114 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d2b8b-be63-4087-ac01-8dbd54a06529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:32.772678 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d2b8b-be63-4087-ac01-8dbd54a06529 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:32.772955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d2b8b-be63-4087-ac01-8dbd54a06529 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:32.773226 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d2b8b-be63-4087-ac01-8dbd54a06529 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:32.773805 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164d2b8b-be63-4087-ac01-8dbd54a06529 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:32.775270 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164d2b8b-be63-4087-ac01-8dbd54a06529 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:32.784430 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-164d2b8b-be63-4087-ac01-8dbd54a06529 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 14:55:32.784430 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:32.784430 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:55:32.784430 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:55:32.784430 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:55:32.784430 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:55:32.784430 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:55:32.784430 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:32.784430 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 14:55:32.784430 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:32.784430 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:55:32.784430 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:32.784430 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:32.784430 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:32.784430 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:32.784430 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:32.784430 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:32.785414 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:32.785414 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:32.785414 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:32.785414 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:32.785414 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:32.785414 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:32.785414 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:55:32.785414 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:55:32.785414 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:55:32.785414 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:55:32.785414 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:32.785414 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:32.785414 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:32.785414 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:32.785414 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:32.785414 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:32.786268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:32.786268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:32.786268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:32.786268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:32.786268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:32.786268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:32.786268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:32.786268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:32.786268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:32.786268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:32.786268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:55:32.786268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:32.786268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 14:55:32.786268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:32.786268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 507/1017] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:55:32 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:32.792778 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef7316-f2c9-4924-989e-061c0d46fb27 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:32.874497 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:32.901793 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef7316-f2c9-4924-989e-061c0d46fb27 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:32.936367 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de21fa-7f5b-44e3-8d65-b9881810d146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:32.937042 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de21fa-7f5b-44e3-8d65-b9881810d146 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:32.937326 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de21fa-7f5b-44e3-8d65-b9881810d146 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:32.937600 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de21fa-7f5b-44e3-8d65-b9881810d146 None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:32.938226 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de21fa-7f5b-44e3-8d65-b9881810d146 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:32.938528 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de21fa-7f5b-44e3-8d65-b9881810d146 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:32.939956 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de21fa-7f5b-44e3-8d65-b9881810d146 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:32.949502 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 511/1018] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 11 14:55:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:32.951524 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d2b8b-be63-4087-ac01-8dbd54a06529 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:33.060692 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d2b8b-be63-4087-ac01-8dbd54a06529 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:33.095819 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dcfa02-a013-4e20-bc45-e9e2b3c3f59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:33.096489 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcfa02-a013-4e20-bc45-e9e2b3c3f59e None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:33.096773 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcfa02-a013-4e20-bc45-e9e2b3c3f59e None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:33.097050 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcfa02-a013-4e20-bc45-e9e2b3c3f59e None admin] PATH_INFO: `/v3/services` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:33.097683 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dcfa02-a013-4e20-bc45-e9e2b3c3f59e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:33.097989 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dcfa02-a013-4e20-bc45-e9e2b3c3f59e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:33.099412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dcfa02-a013-4e20-bc45-e9e2b3c3f59e None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:33.109230 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 508/1019] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 11 14:55:32 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:33.111172 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de21fa-7f5b-44e3-8d65-b9881810d146 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:33.214975 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de21fa-7f5b-44e3-8d65-b9881810d146 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:33.250446 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e6aaba-ecf8-42bb-af1f-95d7fce7f487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:33.251095 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6aaba-ecf8-42bb-af1f-95d7fce7f487 None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:33.251378 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6aaba-ecf8-42bb-af1f-95d7fce7f487 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:33.251650 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6aaba-ecf8-42bb-af1f-95d7fce7f487 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:33.252318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e6aaba-ecf8-42bb-af1f-95d7fce7f487 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:33.253701 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e6aaba-ecf8-42bb-af1f-95d7fce7f487 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:33.263203 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:33.276620 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 512/1020] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 14:55:33 2022] POST /identity/v3/endpoints => generated 334 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 14:55:33.741016 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54b042-8945-4c8a-811d-25a0f4debe57 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:33.741543 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54b042-8945-4c8a-811d-25a0f4debe57 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:33.741543 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54b042-8945-4c8a-811d-25a0f4debe57 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:33.742966 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 509/1021] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:33.748275 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e7df7-c1b9-40bb-8cfe-e91f38efc1ab None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:33.748562 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e7df7-c1b9-40bb-8cfe-e91f38efc1ab None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:33.748838 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e7df7-c1b9-40bb-8cfe-e91f38efc1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:33.767779 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-af2e7df7-c1b9-40bb-8cfe-e91f38efc1ab None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:33.768984 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-af2e7df7-c1b9-40bb-8cfe-e91f38efc1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:33.843839 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 513/1022] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:33 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:34.113238 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d54b042-8945-4c8a-811d-25a0f4debe57 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:34.143374 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:34.220816 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d54b042-8945-4c8a-811d-25a0f4debe57 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:34.255806 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017c222e-7cb5-4639-b588-5a79e671bfbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:34.256368 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c222e-7cb5-4639-b588-5a79e671bfbb None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:34.256642 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c222e-7cb5-4639-b588-5a79e671bfbb None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:34.256919 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c222e-7cb5-4639-b588-5a79e671bfbb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:34.258048 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017c222e-7cb5-4639-b588-5a79e671bfbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:34.259530 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017c222e-7cb5-4639-b588-5a79e671bfbb None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:34.261286 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 510/1023] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:55:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:34.266914 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e7df7-c1b9-40bb-8cfe-e91f38efc1ab None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:34.369740 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e7df7-c1b9-40bb-8cfe-e91f38efc1ab None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:34.394050 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:34.406870 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9978c45a-78de-4491-86ec-a4c6195d6683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:34.407489 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978c45a-78de-4491-86ec-a4c6195d6683 None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:34.407771 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978c45a-78de-4491-86ec-a4c6195d6683 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:34.408046 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978c45a-78de-4491-86ec-a4c6195d6683 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:34.408712 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9978c45a-78de-4491-86ec-a4c6195d6683 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:34.410151 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9978c45a-78de-4491-86ec-a4c6195d6683 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:34.438847 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 514/1024] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 14:55:34 2022] POST /identity/v3/users => generated 285 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:55:34.748664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7dd1b6-21e1-4d9c-9033-a196bbe91474 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:34.749218 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7dd1b6-21e1-4d9c-9033-a196bbe91474 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:34.749218 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7dd1b6-21e1-4d9c-9033-a196bbe91474 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:34.750695 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 511/1025] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:34.755782 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aaa064-7556-4ebf-9187-c3645bd79034 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:34.756075 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aaa064-7556-4ebf-9187-c3645bd79034 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:34.756340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aaa064-7556-4ebf-9187-c3645bd79034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:34.774621 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-64aaa064-7556-4ebf-9187-c3645bd79034 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:34.775846 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-64aaa064-7556-4ebf-9187-c3645bd79034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:34.844477 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 515/1026] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:34 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:35.115119 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7dd1b6-21e1-4d9c-9033-a196bbe91474 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:35.212918 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:35.222531 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7dd1b6-21e1-4d9c-9033-a196bbe91474 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:35.257652 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ce7ab-ca9f-451d-beeb-473ad3c48ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:35.258288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ce7ab-ca9f-451d-beeb-473ad3c48ddc None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:35.258567 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ce7ab-ca9f-451d-beeb-473ad3c48ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:35.258882 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ce7ab-ca9f-451d-beeb-473ad3c48ddc None admin] PATH_INFO: `/v3/services/image` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:35.259467 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ce7ab-ca9f-451d-beeb-473ad3c48ddc None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:35.260827 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ce7ab-ca9f-451d-beeb-473ad3c48ddc None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:35.269633 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-2b2ce7ab-ca9f-451d-beeb-473ad3c48ddc None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 14:55:35.269633 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:35.269633 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:55:35.269633 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:55:35.269633 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:55:35.269633 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:55:35.269633 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:55:35.269633 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:35.269633 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 14:55:35.269633 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:35.269633 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:55:35.269633 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:35.269633 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:35.269633 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:35.269633 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:35.269633 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:35.269633 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:35.270638 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:35.270638 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:35.270638 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:35.270638 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:35.270638 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:35.270638 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:35.270638 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:55:35.270638 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:55:35.270638 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:55:35.270638 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:55:35.270638 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:35.270638 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:35.270638 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:35.270638 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:35.270638 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:35.270638 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:35.271538 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:35.271538 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:35.271538 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:35.271538 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:35.271538 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:35.271538 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:35.271538 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:35.271538 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:35.271538 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:35.271538 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:35.271538 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:55:35.271538 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:35.271538 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 14:55:35.271538 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:35.271538 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 512/1027] 10.222.0.33 () {56 vars in 1105 bytes} [Sun Sep 11 14:55:35 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:35.279085 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aaa064-7556-4ebf-9187-c3645bd79034 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:35.382707 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aaa064-7556-4ebf-9187-c3645bd79034 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:35.393583 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:35.422325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd7bbe-2b4e-4a23-9dc3-a9e38a9b0be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:35.422967 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd7bbe-2b4e-4a23-9dc3-a9e38a9b0be5 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:35.423245 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd7bbe-2b4e-4a23-9dc3-a9e38a9b0be5 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:35.423514 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd7bbe-2b4e-4a23-9dc3-a9e38a9b0be5 None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:35.424133 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd7bbe-2b4e-4a23-9dc3-a9e38a9b0be5 None admin] RBAC: Adding query filter params (name=image) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:35.424431 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd7bbe-2b4e-4a23-9dc3-a9e38a9b0be5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:35.425808 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd7bbe-2b4e-4a23-9dc3-a9e38a9b0be5 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:35.435846 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 516/1028] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:55:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:35.436874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ce7ab-ca9f-451d-beeb-473ad3c48ddc None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:35.543882 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ce7ab-ca9f-451d-beeb-473ad3c48ddc None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:35.578711 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07922c-e092-40f2-8c38-57dd470465ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:35.579347 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07922c-e092-40f2-8c38-57dd470465ef None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:35.579622 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07922c-e092-40f2-8c38-57dd470465ef None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:35.579916 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07922c-e092-40f2-8c38-57dd470465ef None admin] PATH_INFO: `/v3/services` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:35.580645 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d07922c-e092-40f2-8c38-57dd470465ef None admin] RBAC: Adding query filter params (type=image) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:35.580891 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d07922c-e092-40f2-8c38-57dd470465ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:35.582112 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd61a55-2f59-48c3-85f0-75cbce117516 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:35.582279 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d07922c-e092-40f2-8c38-57dd470465ef None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:35.582382 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd61a55-2f59-48c3-85f0-75cbce117516 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:35.582656 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd61a55-2f59-48c3-85f0-75cbce117516 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:35.584156 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 517/1029] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:35.592858 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 513/1030] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:55:35 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:55:35.593002 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1086616-9795-4e9b-bc5b-90e9a8dccafd None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:35.593002 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1086616-9795-4e9b-bc5b-90e9a8dccafd None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:35.593002 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1086616-9795-4e9b-bc5b-90e9a8dccafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:35.593653 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07922c-e092-40f2-8c38-57dd470465ef None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:35.609089 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-b1086616-9795-4e9b-bc5b-90e9a8dccafd None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:35.610382 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-b1086616-9795-4e9b-bc5b-90e9a8dccafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:35.619695 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:35.682912 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 518/1031] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:35 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 14:55:35.703888 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07922c-e092-40f2-8c38-57dd470465ef None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:35.739376 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db53efb-1f92-40b6-8122-f20e7bd37186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:35.739990 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db53efb-1f92-40b6-8122-f20e7bd37186 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:35.740267 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db53efb-1f92-40b6-8122-f20e7bd37186 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:35.740536 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db53efb-1f92-40b6-8122-f20e7bd37186 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:35.741314 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db53efb-1f92-40b6-8122-f20e7bd37186 None admin] RBAC: Adding query filter params (interface=public, service_id=ee1f9455855545acb14dbd0449192dc6, region_id=RegionOne) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:35.741622 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db53efb-1f92-40b6-8122-f20e7bd37186 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:35.743019 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db53efb-1f92-40b6-8122-f20e7bd37186 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:35.753768 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 514/1032] 10.222.0.33 () {56 vars in 1251 bytes} [Sun Sep 11 14:55:35 2022] GET /identity/v3/endpoints?service_id=ee1f9455855545acb14dbd0449192dc6&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:35.754600 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1086616-9795-4e9b-bc5b-90e9a8dccafd None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:35.856257 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1086616-9795-4e9b-bc5b-90e9a8dccafd None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:35.858552 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:35.893909 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81aa24b-143a-4089-b2a3-5f4cb33328af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:35.894521 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81aa24b-143a-4089-b2a3-5f4cb33328af None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:35.894813 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81aa24b-143a-4089-b2a3-5f4cb33328af None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:35.895099 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81aa24b-143a-4089-b2a3-5f4cb33328af None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:35.895703 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81aa24b-143a-4089-b2a3-5f4cb33328af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:35.897109 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81aa24b-143a-4089-b2a3-5f4cb33328af None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:35.903919 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 519/1033] 10.222.0.33 () {56 vars in 1087 bytes} [Sun Sep 11 14:55:35 2022] GET /identity/v3/services => generated 1957 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 14:55:35.956630 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db53efb-1f92-40b6-8122-f20e7bd37186 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:36.054875 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:36.072377 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db53efb-1f92-40b6-8122-f20e7bd37186 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:36.109687 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370bb58-9d1d-4f69-9127-85f9008d14e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:36.110327 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370bb58-9d1d-4f69-9127-85f9008d14e5 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:36.110613 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370bb58-9d1d-4f69-9127-85f9008d14e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:36.110928 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370bb58-9d1d-4f69-9127-85f9008d14e5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:36.121700 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f370bb58-9d1d-4f69-9127-85f9008d14e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:55:36.126798 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-f370bb58-9d1d-4f69-9127-85f9008d14e5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 14:55:36.126798 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:36.126798 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:36.126798 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:36.126798 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:36.126798 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:36.126798 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:36.126798 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:36.126798 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:36.126798 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:36.126798 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:36.126798 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:36.126798 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:55:36.126798 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:55:36.126798 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:55:36.126798 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:55:36.127750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:55:36.127750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:55:36.127750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:55:36.127750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:55:36.127750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:36.127750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:36.127750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:36.127750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:36.127750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:36.127750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:36.127750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:36.127750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:36.127750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:36.127750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:36.127750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:36.127750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:36.128599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:36.128599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:36.128599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:36.128599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:36.128599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:55:36.128599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:55:36.128599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:55:36.128599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:55:36.128599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:55:36.128599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:55:36.128599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 14:55:36.128599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:36.128599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 515/1034] 10.222.0.33 () {56 vars in 1120 bytes} [Sun Sep 11 14:55:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:36.133865 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81aa24b-143a-4089-b2a3-5f4cb33328af None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:36.238237 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81aa24b-143a-4089-b2a3-5f4cb33328af None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:36.272862 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc51c3cd-77da-47d0-8ec2-bb30e84be5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:36.273479 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51c3cd-77da-47d0-8ec2-bb30e84be5e9 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:36.273760 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51c3cd-77da-47d0-8ec2-bb30e84be5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:36.274034 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51c3cd-77da-47d0-8ec2-bb30e84be5e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:36.274705 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc51c3cd-77da-47d0-8ec2-bb30e84be5e9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:36.275029 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc51c3cd-77da-47d0-8ec2-bb30e84be5e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:36.276414 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc51c3cd-77da-47d0-8ec2-bb30e84be5e9 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:36.290092 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 520/1035] 10.222.0.33 () {56 vars in 1115 bytes} [Sun Sep 11 14:55:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:36.291327 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370bb58-9d1d-4f69-9127-85f9008d14e5 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:36.399806 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370bb58-9d1d-4f69-9127-85f9008d14e5 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:36.435958 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbb31fb-790e-4674-84bb-29a4d790d291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:36.436604 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb31fb-790e-4674-84bb-29a4d790d291 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:36.436883 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb31fb-790e-4674-84bb-29a4d790d291 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:36.437154 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb31fb-790e-4674-84bb-29a4d790d291 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:36.447405 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbb31fb-790e-4674-84bb-29a4d790d291 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:36.448865 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbb31fb-790e-4674-84bb-29a4d790d291 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:36.456501 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:36.466254 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-7fbb31fb-790e-4674-84bb-29a4d790d291 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:36.466254 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:36.466254 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:36.466254 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:36.466254 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:36.466254 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:36.466254 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:36.466254 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:36.466254 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:36.466254 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:36.466254 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:36.466254 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:36.466254 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:55:36.466254 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:55:36.466254 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:55:36.466254 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:55:36.472041 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:36.472041 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:36.472041 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:36.472041 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:36.472041 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:36.472041 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:36.472041 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:36.472041 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:36.472041 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:36.472041 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:36.472041 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:36.472041 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:36.472041 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:36.472041 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:36.472041 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:36.472041 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:36.472041 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:55:36.473011 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:55:36.473011 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:36.473011 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:36.473011 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 516/1036] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:55:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:55:36.473242 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc51c3cd-77da-47d0-8ec2-bb30e84be5e9 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:36.502699 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:36.581374 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc51c3cd-77da-47d0-8ec2-bb30e84be5e9 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:36.619684 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969de188-0d70-41e8-8ca3-b5b5a8dcc850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:36.620299 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969de188-0d70-41e8-8ca3-b5b5a8dcc850 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:36.620582 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969de188-0d70-41e8-8ca3-b5b5a8dcc850 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:36.620857 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969de188-0d70-41e8-8ca3-b5b5a8dcc850 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:36.621505 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969de188-0d70-41e8-8ca3-b5b5a8dcc850 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:36.621807 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969de188-0d70-41e8-8ca3-b5b5a8dcc850 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:36.623299 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969de188-0d70-41e8-8ca3-b5b5a8dcc850 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:36.640706 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 521/1037] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:55:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:36.641886 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbb31fb-790e-4674-84bb-29a4d790d291 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:36.709985 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ea053-9f64-44b2-89cf-387bd7e533c3 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:36.710272 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ea053-9f64-44b2-89cf-387bd7e533c3 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:36.710549 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ea053-9f64-44b2-89cf-387bd7e533c3 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:36.712208 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 522/1038] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:55:36.718085 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c96262-a80d-45db-b2e3-483ef9e0e9f0 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:36.718364 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c96262-a80d-45db-b2e3-483ef9e0e9f0 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:36.718637 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c96262-a80d-45db-b2e3-483ef9e0e9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:36.737350 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-06c96262-a80d-45db-b2e3-483ef9e0e9f0 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:36.738538 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-06c96262-a80d-45db-b2e3-483ef9e0e9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:36.753426 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbb31fb-790e-4674-84bb-29a4d790d291 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:36.788250 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f95631-761b-4899-a5ba-e2f027ff5b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:36.789581 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f95631-761b-4899-a5ba-e2f027ff5b51 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:36.790007 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f95631-761b-4899-a5ba-e2f027ff5b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:36.790475 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f95631-761b-4899-a5ba-e2f027ff5b51 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:36.799524 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f95631-761b-4899-a5ba-e2f027ff5b51 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:36.801404 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f95631-761b-4899-a5ba-e2f027ff5b51 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:36.814056 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 523/1039] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 14:55:36.816174 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-f1f95631-761b-4899-a5ba-e2f027ff5b51 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 14:55:36.816174 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:36.816174 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:36.816174 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:36.816174 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:36.816174 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:36.816174 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:36.816174 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:36.816174 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:36.816174 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:36.816174 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:36.816174 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:36.816174 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:55:36.816174 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:55:36.816174 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:55:36.816174 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:55:36.817299 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:36.817299 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:36.817299 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:55:36.817299 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:55:36.817299 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:55:36.817299 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:55:36.817299 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:36.817299 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:36.817299 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:36.817299 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:36.817299 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:36.817299 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:36.817299 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:36.817299 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:36.817299 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:36.817299 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:36.817299 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:36.817299 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:36.818474 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:36.818474 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:36.818474 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:55:36.818474 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:55:36.818474 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:55:36.818474 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:55:36.818474 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:55:36.818474 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:55:36.818474 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 14:55:36.818474 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:36.820613 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 517/1040] 10.222.0.33 () {56 vars in 1093 bytes} [Sun Sep 11 14:55:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:36.826103 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c96262-a80d-45db-b2e3-483ef9e0e9f0 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:36.846105 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:36.933051 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c96262-a80d-45db-b2e3-483ef9e0e9f0 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:36.967699 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d1de1-ae77-4921-b454-320db18e0bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:36.968306 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d1de1-ae77-4921-b454-320db18e0bde None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:36.968585 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d1de1-ae77-4921-b454-320db18e0bde None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:36.968856 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d1de1-ae77-4921-b454-320db18e0bde None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:36.969572 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d1de1-ae77-4921-b454-320db18e0bde None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:36.969874 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d1de1-ae77-4921-b454-320db18e0bde None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:36.971344 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d1de1-ae77-4921-b454-320db18e0bde None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:36.991935 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 524/1041] 10.222.0.33 () {56 vars in 1133 bytes} [Sun Sep 11 14:55:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:55:36.997616 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f95631-761b-4899-a5ba-e2f027ff5b51 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:37.059448 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:37.085299 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d1de1-ae77-4921-b454-320db18e0bde None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:37.107303 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f95631-761b-4899-a5ba-e2f027ff5b51 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:37.144991 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ad2dc-701d-407c-967e-6973e4ad932b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:37.145772 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ad2dc-701d-407c-967e-6973e4ad932b None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:37.146393 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ad2dc-701d-407c-967e-6973e4ad932b None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:37.146675 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ad2dc-701d-407c-967e-6973e4ad932b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:37.168360 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:37.196982 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d1de1-ae77-4921-b454-320db18e0bde None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:37.225010 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333ad2dc-701d-407c-967e-6973e4ad932b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:37.226748 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333ad2dc-701d-407c-967e-6973e4ad932b None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:37.235437 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6906021d-c61b-4007-aca9-0b9db66c2796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:37.235874 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906021d-c61b-4007-aca9-0b9db66c2796 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:37.236150 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906021d-c61b-4007-aca9-0b9db66c2796 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:37.236415 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906021d-c61b-4007-aca9-0b9db66c2796 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:37.247401 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6906021d-c61b-4007-aca9-0b9db66c2796 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:55:37.252551 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-6906021d-c61b-4007-aca9-0b9db66c2796 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:55:37.252551 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:37.252551 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:37.252551 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:37.252551 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:37.252551 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:37.252551 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:37.252551 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:37.252551 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:37.252551 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:37.252551 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:37.252551 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:37.252551 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:55:37.252551 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:55:37.252551 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:55:37.252551 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:55:37.253484 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:55:37.253484 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:55:37.253484 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:55:37.253484 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:55:37.253484 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:37.253484 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:37.253484 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:37.253484 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:37.253484 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:37.253484 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:37.253484 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:37.253484 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:37.253484 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:37.253484 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:37.253484 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:37.253484 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:37.254327 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:37.254327 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:37.254327 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:37.254327 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:37.254327 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:55:37.254327 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:55:37.254327 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:55:37.254327 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:55:37.254327 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:55:37.254327 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:55:37.254327 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:55:37.254327 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:37.254327 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 525/1042] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:55:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:55:37.260003 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6906021d-c61b-4007-aca9-0b9db66c2796 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:37.294488 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:37.346919 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 518/1043] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:55:36 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 14:55:37.348506 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ad2dc-701d-407c-967e-6973e4ad932b None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:37.373533 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6906021d-c61b-4007-aca9-0b9db66c2796 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:37.410018 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dbcac8-59af-40b3-8f0b-1d47a78e5050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:37.410921 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dbcac8-59af-40b3-8f0b-1d47a78e5050 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:37.411208 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dbcac8-59af-40b3-8f0b-1d47a78e5050 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:37.411482 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dbcac8-59af-40b3-8f0b-1d47a78e5050 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:37.412143 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dbcac8-59af-40b3-8f0b-1d47a78e5050 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:37.412444 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dbcac8-59af-40b3-8f0b-1d47a78e5050 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:37.413826 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dbcac8-59af-40b3-8f0b-1d47a78e5050 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:37.416205 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:37.424872 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 526/1044] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:55:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:55:37.430460 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dbcac8-59af-40b3-8f0b-1d47a78e5050 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:37.459108 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ad2dc-701d-407c-967e-6973e4ad932b None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:37.494315 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ba02b-0768-4fcf-ba90-896050d1fa0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:37.495187 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ba02b-0768-4fcf-ba90-896050d1fa0f None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:37.495301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ba02b-0768-4fcf-ba90-896050d1fa0f None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:37.495486 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ba02b-0768-4fcf-ba90-896050d1fa0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:37.506382 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69ba02b-0768-4fcf-ba90-896050d1fa0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:37.507869 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69ba02b-0768-4fcf-ba90-896050d1fa0f None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:37.523910 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-e69ba02b-0768-4fcf-ba90-896050d1fa0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:37.523910 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:37.523910 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:37.523910 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:37.523910 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:37.523910 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:37.523910 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:37.523910 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:37.523910 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:37.523910 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:37.523910 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:37.523910 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:37.523910 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:55:37.523910 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:55:37.523910 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:55:37.523910 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:55:37.524902 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:37.524902 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:37.524902 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:37.524902 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:37.524902 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:37.524902 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:37.524902 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:37.524902 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:37.524902 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:37.524902 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:37.524902 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:37.524902 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:37.524902 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:37.524902 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:37.524902 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:37.524902 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:37.524902 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:55:37.530865 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:55:37.530865 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:37.530865 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:37.530865 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 519/1045] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:55:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:55:37.530865 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ba02b-0768-4fcf-ba90-896050d1fa0f None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:37.546191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:37.547634 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dbcac8-59af-40b3-8f0b-1d47a78e5050 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:37.583607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f5f65-3639-4338-ae93-e047f5a2b5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:37.584000 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f5f65-3639-4338-ae93-e047f5a2b5d8 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:37.584279 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f5f65-3639-4338-ae93-e047f5a2b5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:37.584554 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f5f65-3639-4338-ae93-e047f5a2b5d8 None admin] PATH_INFO: `/v3/users/44f728866dc1443082b86698c151242d` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:37.598796 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f5f65-3639-4338-ae93-e047f5a2b5d8 None admin] RBAC: Authorizing `identity:get_user(user_id=44f728866dc1443082b86698c151242d)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:37.600227 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f5f65-3639-4338-ae93-e047f5a2b5d8 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:37.602196 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 527/1046] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:55:37 2022] GET /identity/v3/users/44f728866dc1443082b86698c151242d => generated 285 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:55:37.608072 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f5f65-3639-4338-ae93-e047f5a2b5d8 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:37.644230 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ba02b-0768-4fcf-ba90-896050d1fa0f None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:37.668274 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:37.681119 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816e4ed-480f-4248-a8a5-574aa37aa78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:37.681971 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816e4ed-480f-4248-a8a5-574aa37aa78e None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:37.682248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816e4ed-480f-4248-a8a5-574aa37aa78e None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:37.682523 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816e4ed-480f-4248-a8a5-574aa37aa78e None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:37.683191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816e4ed-480f-4248-a8a5-574aa37aa78e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:37.683494 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816e4ed-480f-4248-a8a5-574aa37aa78e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:37.685787 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816e4ed-480f-4248-a8a5-574aa37aa78e None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:37.698611 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 520/1047] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:55:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:55:37.704812 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816e4ed-480f-4248-a8a5-574aa37aa78e None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:37.719721 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f5f65-3639-4338-ae93-e047f5a2b5d8 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:37.757979 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce716bdd-4b1c-4a9a-9311-e8c86deb8ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:37.759085 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce716bdd-4b1c-4a9a-9311-e8c86deb8ed1 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:37.759812 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce716bdd-4b1c-4a9a-9311-e8c86deb8ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:37.760297 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce716bdd-4b1c-4a9a-9311-e8c86deb8ed1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:37.780999 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:37.815853 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816e4ed-480f-4248-a8a5-574aa37aa78e None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:37.832573 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce716bdd-4b1c-4a9a-9311-e8c86deb8ed1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:37.834036 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce716bdd-4b1c-4a9a-9311-e8c86deb8ed1 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:37.853179 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5d17f-4ce0-42dc-8a0b-92c86967c765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:37.853727 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5d17f-4ce0-42dc-8a0b-92c86967c765 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:37.854008 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5d17f-4ce0-42dc-8a0b-92c86967c765 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:37.854282 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5d17f-4ce0-42dc-8a0b-92c86967c765 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:37.861845 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e5d17f-4ce0-42dc-8a0b-92c86967c765 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:37.863333 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e5d17f-4ce0-42dc-8a0b-92c86967c765 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:37.876427 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-12e5d17f-4ce0-42dc-8a0b-92c86967c765 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:55:37.876427 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:37.876427 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:37.876427 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:37.876427 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:37.876427 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:37.876427 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:37.876427 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:37.876427 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:37.876427 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:37.876427 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:37.876427 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:37.876427 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:55:37.876427 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:55:37.876427 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:55:37.877334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:55:37.877334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:37.877334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:37.877334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:37.877334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:37.877334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:37.877334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:37.877334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:37.877334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:37.877334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:37.877334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:37.877334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:37.877334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:37.877334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:37.877334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:37.877334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:37.877334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:37.878995 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:55:37.878995 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:55:37.878995 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:55:37.878995 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:55:37.878995 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:55:37.878995 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:55:37.878995 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:55:37.878995 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:37.878995 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 521/1048] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:55:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:55:37.883214 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5d17f-4ce0-42dc-8a0b-92c86967c765 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:37.897721 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:37.947027 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 528/1049] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:55:37 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 14:55:37.953190 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce716bdd-4b1c-4a9a-9311-e8c86deb8ed1 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:38.001552 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5d17f-4ce0-42dc-8a0b-92c86967c765 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:38.035920 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:38.043653 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6de746-8a7e-4e15-9af8-27410e73c807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:38.044322 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6de746-8a7e-4e15-9af8-27410e73c807 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:38.044606 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6de746-8a7e-4e15-9af8-27410e73c807 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:38.044863 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6de746-8a7e-4e15-9af8-27410e73c807 None admin] PATH_INFO: `/v3/projects` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:38.045559 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6de746-8a7e-4e15-9af8-27410e73c807 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:38.045867 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6de746-8a7e-4e15-9af8-27410e73c807 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:38.047343 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6de746-8a7e-4e15-9af8-27410e73c807 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:38.065146 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 522/1050] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:55:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:55:38.065344 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce716bdd-4b1c-4a9a-9311-e8c86deb8ed1 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:38.066211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6de746-8a7e-4e15-9af8-27410e73c807 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:38.104482 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f389f8-acc3-472a-bd64-64c2882d8ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:38.105037 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f389f8-acc3-472a-bd64-64c2882d8ee8 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:38.105317 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f389f8-acc3-472a-bd64-64c2882d8ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:38.105595 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f389f8-acc3-472a-bd64-64c2882d8ee8 None admin] PATH_INFO: `/v3/projects/829f2ddf22294cdd929f2edf944acfe8` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:38.106732 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f389f8-acc3-472a-bd64-64c2882d8ee8 None admin] RBAC: Authorizing `identity:get_project(project_id=829f2ddf22294cdd929f2edf944acfe8)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:38.108386 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f389f8-acc3-472a-bd64-64c2882d8ee8 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:38.110842 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 529/1051] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:55:37 2022] GET /identity/v3/projects/829f2ddf22294cdd929f2edf944acfe8 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:55:38.111348 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da08c5-c1b4-42d0-8ab4-8986c0452df7 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:38.111633 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da08c5-c1b4-42d0-8ab4-8986c0452df7 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:38.111901 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da08c5-c1b4-42d0-8ab4-8986c0452df7 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:38.113087 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 530/1052] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:38.117898 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da08c5-c1b4-42d0-8ab4-8986c0452df7 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:38.143138 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:38.174106 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6de746-8a7e-4e15-9af8-27410e73c807 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:38.208251 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2871e-c1b8-430e-accd-2f454a6e93dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:38.208946 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2871e-c1b8-430e-accd-2f454a6e93dd None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:38.209221 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2871e-c1b8-430e-accd-2f454a6e93dd None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:38.209492 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2871e-c1b8-430e-accd-2f454a6e93dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:38.210202 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f2871e-c1b8-430e-accd-2f454a6e93dd None admin] RBAC: Adding query filter params (role.id=56ed900824b74d938089a39aeb349cf1, scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176, user.id=fc0ea93084b0482fb2dd6d802987942f) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:38.210507 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f2871e-c1b8-430e-accd-2f454a6e93dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:38.211985 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f2871e-c1b8-430e-accd-2f454a6e93dd None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:38.218209 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 523/1053] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:55:38 2022] GET /identity/v3/role_assignments?user.id=fc0ea93084b0482fb2dd6d802987942f&scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176&role.id=56ed900824b74d938089a39aeb349cf1 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:38.219682 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48e767-bcf8-49ed-9480-fd48a6fda274 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:38.220173 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48e767-bcf8-49ed-9480-fd48a6fda274 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:38.220537 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48e767-bcf8-49ed-9480-fd48a6fda274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:38.220651 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da08c5-c1b4-42d0-8ab4-8986c0452df7 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:38.242475 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-9a48e767-bcf8-49ed-9480-fd48a6fda274 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:38.243981 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-9a48e767-bcf8-49ed-9480-fd48a6fda274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:38.261145 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f24834-de63-4d06-9c39-b373793b4f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:38.261762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f24834-de63-4d06-9c39-b373793b4f5a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:38.262044 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f24834-de63-4d06-9c39-b373793b4f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:38.262322 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f24834-de63-4d06-9c39-b373793b4f5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:38.263877 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f24834-de63-4d06-9c39-b373793b4f5a None admin] RBAC: Adding query filter params (role.id=99e8fe7df8e7446680a81d4eaaa1cb68, scope.project.id=829f2ddf22294cdd929f2edf944acfe8, user.id=44f728866dc1443082b86698c151242d) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:38.264258 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f24834-de63-4d06-9c39-b373793b4f5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:38.265731 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f24834-de63-4d06-9c39-b373793b4f5a None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:38.271772 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 531/1054] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:55:38 2022] GET /identity/v3/role_assignments?user.id=44f728866dc1443082b86698c151242d&scope.project.id=829f2ddf22294cdd929f2edf944acfe8&role.id=99e8fe7df8e7446680a81d4eaaa1cb68 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:38.320019 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:38.342531 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 524/1055] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:38.618906 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f24834-de63-4d06-9c39-b373793b4f5a None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:38.712286 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:38.725933 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f24834-de63-4d06-9c39-b373793b4f5a None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:38.762493 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ccb5d7-b87e-4cd5-a76e-c704881b530c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:38.763210 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ccb5d7-b87e-4cd5-a76e-c704881b530c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:38.763488 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ccb5d7-b87e-4cd5-a76e-c704881b530c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:38.763759 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ccb5d7-b87e-4cd5-a76e-c704881b530c None admin] PATH_INFO: `/v3/services/image` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:38.764338 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ccb5d7-b87e-4cd5-a76e-c704881b530c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:38.765714 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ccb5d7-b87e-4cd5-a76e-c704881b530c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:38.774535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-90ccb5d7-b87e-4cd5-a76e-c704881b530c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 14:55:38.774535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:38.774535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:55:38.774535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:55:38.774535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:55:38.774535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:55:38.774535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:55:38.774535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:38.774535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 14:55:38.774535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:38.774535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:55:38.774535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:38.774535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:38.774535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:38.774535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:38.774535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:38.774535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:38.775575 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:38.775575 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:38.775575 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:38.775575 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:38.775575 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:38.775575 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:38.775575 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:55:38.775575 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:55:38.775575 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:55:38.775575 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:55:38.775575 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:38.775575 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:38.775575 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:38.775575 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:38.775575 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:38.775575 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:38.776972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:38.776972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:38.776972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:38.776972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:38.776972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:38.776972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:38.776972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:38.776972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:38.776972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:38.776972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:38.776972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:55:38.776972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:55:38.776972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 14:55:38.776972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:38.779198 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 532/1056] 10.222.0.33 () {56 vars in 1105 bytes} [Sun Sep 11 14:55:38 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:38.781571 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a48e767-bcf8-49ed-9480-fd48a6fda274 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:38.884909 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a48e767-bcf8-49ed-9480-fd48a6fda274 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:38.921282 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7459ac-e3ed-465c-b8e9-8f76b82478db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:38.921895 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7459ac-e3ed-465c-b8e9-8f76b82478db None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:38.922198 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7459ac-e3ed-465c-b8e9-8f76b82478db None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:38.922472 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7459ac-e3ed-465c-b8e9-8f76b82478db None admin] PATH_INFO: `/v3/services` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:38.923129 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7459ac-e3ed-465c-b8e9-8f76b82478db None admin] RBAC: Adding query filter params (name=image) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:38.923427 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7459ac-e3ed-465c-b8e9-8f76b82478db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:38.924805 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7459ac-e3ed-465c-b8e9-8f76b82478db None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:38.935010 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 525/1057] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:55:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:38.936087 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ccb5d7-b87e-4cd5-a76e-c704881b530c None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:39.050705 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ccb5d7-b87e-4cd5-a76e-c704881b530c None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:39.086159 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c02e30-e862-49a7-8722-271b5dc4dd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:39.086762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c02e30-e862-49a7-8722-271b5dc4dd93 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:39.087070 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c02e30-e862-49a7-8722-271b5dc4dd93 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:39.087343 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c02e30-e862-49a7-8722-271b5dc4dd93 None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:39.087974 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c02e30-e862-49a7-8722-271b5dc4dd93 None admin] RBAC: Adding query filter params (type=image) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:39.088273 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c02e30-e862-49a7-8722-271b5dc4dd93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:39.089654 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c02e30-e862-49a7-8722-271b5dc4dd93 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:39.095142 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 533/1058] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:55:38 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:39.100876 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7459ac-e3ed-465c-b8e9-8f76b82478db None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:39.154770 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:39.207184 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7459ac-e3ed-465c-b8e9-8f76b82478db None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:39.242206 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf3fea-2e1b-4156-859d-e83817c50f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:39.242831 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf3fea-2e1b-4156-859d-e83817c50f6b None admin] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:39.243116 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf3fea-2e1b-4156-859d-e83817c50f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:39.243388 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf3fea-2e1b-4156-859d-e83817c50f6b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:39.244059 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf3fea-2e1b-4156-859d-e83817c50f6b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:39.245425 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf3fea-2e1b-4156-859d-e83817c50f6b None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:39.270842 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 526/1059] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 14:55:39 2022] POST /identity/v3/endpoints => generated 327 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:55:40.394911 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79ba2e-83ff-48db-8d57-829cd7a0447d None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:40.395433 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79ba2e-83ff-48db-8d57-829cd7a0447d None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:40.395433 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79ba2e-83ff-48db-8d57-829cd7a0447d None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:40.396775 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 534/1060] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:40.401804 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef000e38-13fe-47b0-b261-a4721cd35554 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:40.402201 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef000e38-13fe-47b0-b261-a4721cd35554 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:40.402390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef000e38-13fe-47b0-b261-a4721cd35554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:40.420726 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19645af1-46ea-4e9b-804d-832347867b39 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:40.421051 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19645af1-46ea-4e9b-804d-832347867b39 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:40.421162 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-ef000e38-13fe-47b0-b261-a4721cd35554 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:40.421328 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19645af1-46ea-4e9b-804d-832347867b39 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:40.422354 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-ef000e38-13fe-47b0-b261-a4721cd35554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:40.426279 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 535/1061] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:40.428760 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1091cc-1969-4d31-8aa4-e6a7e698e77a None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:40.429046 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1091cc-1969-4d31-8aa4-e6a7e698e77a None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:40.429316 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1091cc-1969-4d31-8aa4-e6a7e698e77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:40.448903 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-3d1091cc-1969-4d31-8aa4-e6a7e698e77a None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:40.450076 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-3d1091cc-1969-4d31-8aa4-e6a7e698e77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:40.461840 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:40.505780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 527/1062] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:40.526949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 536/1063] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 14:55:40.779205 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef000e38-13fe-47b0-b261-a4721cd35554 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:40.799273 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1091cc-1969-4d31-8aa4-e6a7e698e77a None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:40.811256 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:40.892552 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef000e38-13fe-47b0-b261-a4721cd35554 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:40.916265 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1091cc-1969-4d31-8aa4-e6a7e698e77a None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:40.928942 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c1c396-2a3e-4284-923c-e0ac5df879e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:40.929580 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1c396-2a3e-4284-923c-e0ac5df879e4 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:40.929859 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1c396-2a3e-4284-923c-e0ac5df879e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:40.930130 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1c396-2a3e-4284-923c-e0ac5df879e4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:40.941306 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c1c396-2a3e-4284-923c-e0ac5df879e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:55:40.946329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-93c1c396-2a3e-4284-923c-e0ac5df879e4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 14:55:40.946329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:40.946329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:40.946329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:40.946329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:40.946329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:40.946329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:40.946329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:40.946329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:40.946329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:40.946329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:40.946329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:40.946329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:55:40.946329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:55:40.946329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:55:40.946329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:55:40.947460 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:55:40.947460 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:55:40.947460 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:55:40.947460 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:55:40.947460 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:40.947460 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:40.947460 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:40.947460 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:40.947460 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:40.947460 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:40.947460 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:40.947460 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:40.947460 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:40.947460 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:40.947460 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:40.947460 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:40.948375 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:40.948375 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:40.948375 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:40.948375 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:40.948375 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:55:40.948375 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:55:40.948375 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:55:40.948375 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:55:40.948375 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:55:40.948375 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:55:40.948375 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 14:55:40.948375 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:40.948375 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 528/1064] 10.222.0.33 () {56 vars in 1120 bytes} [Sun Sep 11 14:55:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:40.953592 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c1c396-2a3e-4284-923c-e0ac5df879e4 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:40.958477 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856acee7-25a2-42c5-b5c1-dc733d47e27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:40.959149 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856acee7-25a2-42c5-b5c1-dc733d47e27a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:40.959433 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856acee7-25a2-42c5-b5c1-dc733d47e27a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:40.959708 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856acee7-25a2-42c5-b5c1-dc733d47e27a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:40.971522 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856acee7-25a2-42c5-b5c1-dc733d47e27a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:55:40.976391 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-856acee7-25a2-42c5-b5c1-dc733d47e27a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:55:40.976391 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:40.976391 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:40.976391 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:40.976391 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:40.976391 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:40.976391 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:40.976391 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:40.976391 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:40.976391 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:40.976391 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:40.976391 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:40.976391 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:55:40.976391 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:55:40.976391 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:55:40.976391 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:55:40.977444 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:55:40.977444 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:55:40.977444 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:55:40.977444 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:55:40.977444 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:40.977444 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:40.977444 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:40.977444 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:40.977444 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:40.977444 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:40.977444 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:40.977444 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:40.977444 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:40.977444 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:40.977444 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:40.977444 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:40.978298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:40.978298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:40.978298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:40.978298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:40.978298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:55:40.978298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:55:40.978298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:55:40.978298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:55:40.978298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:55:40.978298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:55:40.978298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:55:40.978298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:40.978298 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 537/1065] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:55:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:55:40.984132 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856acee7-25a2-42c5-b5c1-dc733d47e27a None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:40.994434 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:41.062216 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:41.076062 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c1c396-2a3e-4284-923c-e0ac5df879e4 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:41.096417 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856acee7-25a2-42c5-b5c1-dc733d47e27a None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:41.114304 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2bbf1-2908-4a9b-ad58-f1c031bf1234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:41.114976 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2bbf1-2908-4a9b-ad58-f1c031bf1234 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:41.115257 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2bbf1-2908-4a9b-ad58-f1c031bf1234 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:41.115530 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2bbf1-2908-4a9b-ad58-f1c031bf1234 None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:41.116194 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e2bbf1-2908-4a9b-ad58-f1c031bf1234 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:41.116494 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e2bbf1-2908-4a9b-ad58-f1c031bf1234 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:41.117865 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e2bbf1-2908-4a9b-ad58-f1c031bf1234 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:41.130966 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 529/1066] 10.222.0.33 () {56 vars in 1115 bytes} [Sun Sep 11 14:55:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:41.132047 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2bbf1-2908-4a9b-ad58-f1c031bf1234 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:41.140049 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0471a814-3bd5-4543-ba14-144fdc2cbec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:41.140666 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471a814-3bd5-4543-ba14-144fdc2cbec3 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:41.140950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471a814-3bd5-4543-ba14-144fdc2cbec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:41.141226 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471a814-3bd5-4543-ba14-144fdc2cbec3 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:41.142304 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0471a814-3bd5-4543-ba14-144fdc2cbec3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:41.142617 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0471a814-3bd5-4543-ba14-144fdc2cbec3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:41.144401 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0471a814-3bd5-4543-ba14-144fdc2cbec3 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:41.153339 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 538/1067] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:55:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:55:41.159208 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0471a814-3bd5-4543-ba14-144fdc2cbec3 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:41.239570 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2bbf1-2908-4a9b-ad58-f1c031bf1234 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:41.241047 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:41.277802 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd6730-16d7-4fd4-a25b-14bd990bf5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:41.278491 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd6730-16d7-4fd4-a25b-14bd990bf5dd None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:41.279367 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd6730-16d7-4fd4-a25b-14bd990bf5dd None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:41.279670 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd6730-16d7-4fd4-a25b-14bd990bf5dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:41.279993 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0471a814-3bd5-4543-ba14-144fdc2cbec3 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:41.291180 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cd6730-16d7-4fd4-a25b-14bd990bf5dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:41.292965 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cd6730-16d7-4fd4-a25b-14bd990bf5dd None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:41.308453 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-e2cd6730-16d7-4fd4-a25b-14bd990bf5dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:41.308453 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:41.308453 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:41.308453 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:41.308453 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:41.308453 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:41.308453 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:41.308453 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:41.308453 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:41.308453 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:41.308453 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:41.308453 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:41.308453 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:55:41.308453 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:55:41.308453 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:55:41.308453 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:55:41.310508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:41.310508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:41.310508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:41.310508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:41.310508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:41.310508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:41.310508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:41.310508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:41.310508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:41.310508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:41.310508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:41.310508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:41.310508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:41.310508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:41.310508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:41.310508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:41.310508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:55:41.311725 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:55:41.311725 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:41.311725 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:41.311725 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 530/1068] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:55:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:41.316079 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd6730-16d7-4fd4-a25b-14bd990bf5dd None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:41.320053 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:41.321334 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d1a41-23c4-45c9-9605-930c7dc5ef90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:41.322141 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d1a41-23c4-45c9-9605-930c7dc5ef90 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:41.322676 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d1a41-23c4-45c9-9605-930c7dc5ef90 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:41.323012 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d1a41-23c4-45c9-9605-930c7dc5ef90 None admin] PATH_INFO: `/v3/users/44f728866dc1443082b86698c151242d` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:41.324640 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d1a41-23c4-45c9-9605-930c7dc5ef90 None admin] RBAC: Authorizing `identity:get_user(user_id=44f728866dc1443082b86698c151242d)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:41.326582 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d1a41-23c4-45c9-9605-930c7dc5ef90 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:41.333626 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 539/1069] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:55:41 2022] GET /identity/v3/users/44f728866dc1443082b86698c151242d => generated 285 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:41.335005 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d1a41-23c4-45c9-9605-930c7dc5ef90 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:41.430902 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd6730-16d7-4fd4-a25b-14bd990bf5dd None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:41.444293 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d1a41-23c4-45c9-9605-930c7dc5ef90 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:41.469112 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4433c4-7a48-4f2b-85d8-d4a060efa491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:41.469723 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4433c4-7a48-4f2b-85d8-d4a060efa491 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:41.470003 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4433c4-7a48-4f2b-85d8-d4a060efa491 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:41.470286 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4433c4-7a48-4f2b-85d8-d4a060efa491 None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:41.470947 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4433c4-7a48-4f2b-85d8-d4a060efa491 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:41.471254 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4433c4-7a48-4f2b-85d8-d4a060efa491 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:41.472631 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4433c4-7a48-4f2b-85d8-d4a060efa491 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:41.478826 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8eb48-3fea-49f1-b114-a3a58692e0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:41.479452 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8eb48-3fea-49f1-b114-a3a58692e0ea None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:41.479734 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8eb48-3fea-49f1-b114-a3a58692e0ea None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:41.480007 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8eb48-3fea-49f1-b114-a3a58692e0ea None admin] PATH_INFO: `/v3/projects/829f2ddf22294cdd929f2edf944acfe8` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:41.481116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8eb48-3fea-49f1-b114-a3a58692e0ea None admin] RBAC: Authorizing `identity:get_project(project_id=829f2ddf22294cdd929f2edf944acfe8)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:41.482698 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8eb48-3fea-49f1-b114-a3a58692e0ea None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:41.484503 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 540/1070] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:55:41 2022] GET /identity/v3/projects/829f2ddf22294cdd929f2edf944acfe8 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:41.489528 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 531/1071] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:55:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:55:41.489677 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4339d-d910-45ed-9ef8-f2b37c66fe1f None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:41.489677 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4339d-d910-45ed-9ef8-f2b37c66fe1f None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:41.489677 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4339d-d910-45ed-9ef8-f2b37c66fe1f None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:41.489677 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 541/1072] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:41.490092 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4433c4-7a48-4f2b-85d8-d4a060efa491 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:41.490197 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e4339d-d910-45ed-9ef8-f2b37c66fe1f None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:41.494772 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:41.552409 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:41.592533 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4433c4-7a48-4f2b-85d8-d4a060efa491 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:41.595606 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e4339d-d910-45ed-9ef8-f2b37c66fe1f None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:41.628861 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e93f040-578b-4127-88ae-d1421723c094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:41.629518 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93f040-578b-4127-88ae-d1421723c094 None admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:41.629797 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93f040-578b-4127-88ae-d1421723c094 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:41.630066 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93f040-578b-4127-88ae-d1421723c094 None admin] PATH_INFO: `/v3/projects/829f2ddf22294cdd929f2edf944acfe8/users/44f728866dc1443082b86698c151242d/roles/99e8fe7df8e7446680a81d4eaaa1cb68` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:41.631988 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e93f040-578b-4127-88ae-d1421723c094 None admin] RBAC: Authorizing `identity:create_grant(project_id=829f2ddf22294cdd929f2edf944acfe8, user_id=44f728866dc1443082b86698c151242d, role_id=99e8fe7df8e7446680a81d4eaaa1cb68)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:41.632188 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d7316-4465-4d7b-9e2c-7d5efb96dba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:41.632817 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d7316-4465-4d7b-9e2c-7d5efb96dba8 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:41.633093 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d7316-4465-4d7b-9e2c-7d5efb96dba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:41.633364 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d7316-4465-4d7b-9e2c-7d5efb96dba8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:41.633466 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e93f040-578b-4127-88ae-d1421723c094 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:41.641721 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d7316-4465-4d7b-9e2c-7d5efb96dba8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:41.643220 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d7316-4465-4d7b-9e2c-7d5efb96dba8 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:41.646621 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 542/1073] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 14:55:41 2022] PUT /identity/v3/projects/829f2ddf22294cdd929f2edf944acfe8/users/44f728866dc1443082b86698c151242d/roles/99e8fe7df8e7446680a81d4eaaa1cb68 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 14:55:41.647916 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2aca3-9a3e-4207-93ea-dfa3e4457ef4 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:41.648206 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2aca3-9a3e-4207-93ea-dfa3e4457ef4 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:41.648482 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2aca3-9a3e-4207-93ea-dfa3e4457ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:41.655714 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-0d9d7316-4465-4d7b-9e2c-7d5efb96dba8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 14:55:41.655714 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:41.655714 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:41.655714 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:41.655714 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:41.655714 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:41.655714 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:41.655714 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:41.655714 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:41.655714 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:41.655714 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:41.655714 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:41.655714 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:55:41.655714 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:55:41.655714 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:55:41.655714 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:55:41.656785 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:41.656785 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:41.656785 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:55:41.656785 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:55:41.656785 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:55:41.656785 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:55:41.656785 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:41.656785 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:41.656785 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:41.656785 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:41.656785 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:41.656785 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:41.656785 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:41.656785 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:41.656785 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:41.656785 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:41.656785 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:41.656785 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:41.657796 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:41.657796 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:41.657796 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:55:41.657796 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:55:41.657796 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:55:41.657796 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:55:41.657796 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:55:41.657796 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:55:41.657796 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 14:55:41.657796 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:41.657796 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 532/1074] 10.222.0.33 () {56 vars in 1093 bytes} [Sun Sep 11 14:55:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:55:41.663161 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d7316-4465-4d7b-9e2c-7d5efb96dba8 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:41.672466 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-8ec2aca3-9a3e-4207-93ea-dfa3e4457ef4 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:41.673723 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-8ec2aca3-9a3e-4207-93ea-dfa3e4457ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:41.742041 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:41.755013 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 543/1075] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:41.790324 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d7316-4465-4d7b-9e2c-7d5efb96dba8 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:41.816476 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:41.830761 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde7fd5-8c1b-4eaf-9499-a2b3c3ec36fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:41.831404 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde7fd5-8c1b-4eaf-9499-a2b3c3ec36fa None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:41.831686 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde7fd5-8c1b-4eaf-9499-a2b3c3ec36fa None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:41.831961 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde7fd5-8c1b-4eaf-9499-a2b3c3ec36fa None admin] PATH_INFO: `/v3/users` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:41.832691 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde7fd5-8c1b-4eaf-9499-a2b3c3ec36fa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:41.832995 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde7fd5-8c1b-4eaf-9499-a2b3c3ec36fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:41.834434 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde7fd5-8c1b-4eaf-9499-a2b3c3ec36fa None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:41.853338 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 533/1076] 10.222.0.33 () {56 vars in 1133 bytes} [Sun Sep 11 14:55:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:55:41.859095 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2aca3-9a3e-4207-93ea-dfa3e4457ef4 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:41.961985 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2aca3-9a3e-4207-93ea-dfa3e4457ef4 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:42.005566 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf8e67-0722-4a03-962c-e5fa48d21e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:42.006202 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf8e67-0722-4a03-962c-e5fa48d21e0a None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:42.006486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf8e67-0722-4a03-962c-e5fa48d21e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:42.006755 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf8e67-0722-4a03-962c-e5fa48d21e0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:42.017553 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cf8e67-0722-4a03-962c-e5fa48d21e0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:42.019051 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cf8e67-0722-4a03-962c-e5fa48d21e0a None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:42.024157 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde7fd5-8c1b-4eaf-9499-a2b3c3ec36fa None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:42.035116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-d3cf8e67-0722-4a03-962c-e5fa48d21e0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:42.035116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:42.035116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:42.035116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:42.035116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:42.035116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:42.035116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:42.035116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:42.035116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:42.035116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:42.035116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:42.035116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:42.035116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:55:42.035116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:55:42.035116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:55:42.035116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:55:42.036114 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:42.036114 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:42.036114 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:42.036114 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:42.036114 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:42.036114 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:42.036114 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:42.036114 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:42.036114 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:42.036114 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:42.036114 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:42.036114 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:42.036114 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:42.036114 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:42.036114 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:42.036114 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:42.036114 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:55:42.037071 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:55:42.037071 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:42.037071 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:42.037071 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 544/1077] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:55:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:42.043125 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf8e67-0722-4a03-962c-e5fa48d21e0a None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:42.092827 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:42.130668 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde7fd5-8c1b-4eaf-9499-a2b3c3ec36fa None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:42.149181 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf8e67-0722-4a03-962c-e5fa48d21e0a None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:42.169006 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95775043-cbb0-4f86-8ec9-b07f9559ad89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:42.169592 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95775043-cbb0-4f86-8ec9-b07f9559ad89 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:42.169989 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95775043-cbb0-4f86-8ec9-b07f9559ad89 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:42.170264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95775043-cbb0-4f86-8ec9-b07f9559ad89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:42.185723 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792b44f-f726-4ab5-b959-2371fa1278d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:42.186327 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792b44f-f726-4ab5-b959-2371fa1278d2 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:42.186624 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792b44f-f726-4ab5-b959-2371fa1278d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:42.186895 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792b44f-f726-4ab5-b959-2371fa1278d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:42.187673 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9792b44f-f726-4ab5-b959-2371fa1278d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:42.187845 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9792b44f-f726-4ab5-b959-2371fa1278d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:42.189208 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9792b44f-f726-4ab5-b959-2371fa1278d2 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:42.202896 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 545/1078] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:55:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:55:42.208369 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792b44f-f726-4ab5-b959-2371fa1278d2 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:42.229169 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:42.251300 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95775043-cbb0-4f86-8ec9-b07f9559ad89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:42.252754 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95775043-cbb0-4f86-8ec9-b07f9559ad89 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:42.309381 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792b44f-f726-4ab5-b959-2371fa1278d2 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:42.318106 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:42.346731 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d3730-70d2-4ab6-acbf-af1a0f754035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:42.347464 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d3730-70d2-4ab6-acbf-af1a0f754035 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:42.347746 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d3730-70d2-4ab6-acbf-af1a0f754035 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:42.348018 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d3730-70d2-4ab6-acbf-af1a0f754035 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:42.354903 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573d3730-70d2-4ab6-acbf-af1a0f754035 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:42.356366 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573d3730-70d2-4ab6-acbf-af1a0f754035 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:42.368144 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-573d3730-70d2-4ab6-acbf-af1a0f754035 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:55:42.368144 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:42.368144 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:42.368144 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:42.368144 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:42.368144 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:42.368144 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:42.368144 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:42.368144 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:42.368144 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:42.368144 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:42.368144 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:42.368144 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:55:42.368144 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:55:42.368144 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:55:42.369195 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:55:42.369195 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:42.369195 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:42.369195 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:42.369195 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:42.369195 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:42.369195 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:42.369195 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:42.369195 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:42.369195 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:42.369195 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:42.369195 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:42.369195 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:42.369195 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:42.369195 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:42.369195 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:42.369195 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:42.371611 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:55:42.371611 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:55:42.371611 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:55:42.371611 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:55:42.371611 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:55:42.371611 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:55:42.371611 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:55:42.371611 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:42.371611 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 546/1079] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:55:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:55:42.372699 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 534/1080] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:55:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 14:55:42.375493 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d3730-70d2-4ab6-acbf-af1a0f754035 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:42.376242 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95775043-cbb0-4f86-8ec9-b07f9559ad89 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:42.461919 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:42.483303 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95775043-cbb0-4f86-8ec9-b07f9559ad89 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:42.485001 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d3730-70d2-4ab6-acbf-af1a0f754035 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:42.521043 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960a074-a21c-4790-85c0-4e42becb78d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:42.521682 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960a074-a21c-4790-85c0-4e42becb78d2 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:42.521966 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960a074-a21c-4790-85c0-4e42becb78d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:42.522206 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b09474-d4c4-42c5-810e-e8d56032bb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:42.522320 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960a074-a21c-4790-85c0-4e42becb78d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:42.522834 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b09474-d4c4-42c5-810e-e8d56032bb19 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:42.523143 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b09474-d4c4-42c5-810e-e8d56032bb19 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:42.523415 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b09474-d4c4-42c5-810e-e8d56032bb19 None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:42.524173 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b09474-d4c4-42c5-810e-e8d56032bb19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:42.524407 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b09474-d4c4-42c5-810e-e8d56032bb19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:42.525852 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b09474-d4c4-42c5-810e-e8d56032bb19 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:42.533991 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a960a074-a21c-4790-85c0-4e42becb78d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:42.535561 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a960a074-a21c-4790-85c0-4e42becb78d2 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:42.545256 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 547/1081] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:55:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:42.548262 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b09474-d4c4-42c5-810e-e8d56032bb19 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:42.555797 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-a960a074-a21c-4790-85c0-4e42becb78d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:42.555797 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:42.555797 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:42.555797 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:42.555797 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:42.555797 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:42.555797 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:42.555797 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:42.555797 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:42.555797 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:42.555797 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:42.555797 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:42.555797 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:55:42.555797 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:55:42.555797 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:55:42.555797 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:55:42.556768 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:42.556768 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:42.556768 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:42.556768 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:42.556768 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:42.556768 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:42.556768 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:42.556768 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:42.556768 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:42.556768 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:42.556768 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:42.556768 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:42.556768 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:42.556768 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:42.556768 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:42.556768 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:42.556768 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:55:42.562068 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:55:42.562068 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:42.562068 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:42.562068 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 535/1082] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:55:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:55:42.566414 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960a074-a21c-4790-85c0-4e42becb78d2 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:42.578746 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:42.667665 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b09474-d4c4-42c5-810e-e8d56032bb19 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:42.669603 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960a074-a21c-4790-85c0-4e42becb78d2 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:42.705824 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a77ef6-1f53-4cc9-9b7b-03dee3a7cfd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:42.706480 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a77ef6-1f53-4cc9-9b7b-03dee3a7cfd6 None admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:42.706759 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a77ef6-1f53-4cc9-9b7b-03dee3a7cfd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:42.706966 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa5ebb-295e-4e4b-b20a-21d20e4dbf0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:42.707103 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a77ef6-1f53-4cc9-9b7b-03dee3a7cfd6 None admin] PATH_INFO: `/v3/projects/9e4a3d44808e4b4092cfc4ee284c0176/users/fc0ea93084b0482fb2dd6d802987942f/roles/56ed900824b74d938089a39aeb349cf1` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:42.707494 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa5ebb-295e-4e4b-b20a-21d20e4dbf0e None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:42.707774 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa5ebb-295e-4e4b-b20a-21d20e4dbf0e None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:42.708050 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa5ebb-295e-4e4b-b20a-21d20e4dbf0e None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:42.708687 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa5ebb-295e-4e4b-b20a-21d20e4dbf0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:42.708990 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa5ebb-295e-4e4b-b20a-21d20e4dbf0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:42.709555 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a77ef6-1f53-4cc9-9b7b-03dee3a7cfd6 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e4a3d44808e4b4092cfc4ee284c0176, user_id=fc0ea93084b0482fb2dd6d802987942f, role_id=56ed900824b74d938089a39aeb349cf1)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:42.710370 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa5ebb-295e-4e4b-b20a-21d20e4dbf0e None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:42.711015 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a77ef6-1f53-4cc9-9b7b-03dee3a7cfd6 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:42.712072 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:42.725187 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 536/1083] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:55:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:55:42.734855 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 548/1084] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 14:55:42 2022] PUT /identity/v3/projects/9e4a3d44808e4b4092cfc4ee284c0176/users/fc0ea93084b0482fb2dd6d802987942f/roles/56ed900824b74d938089a39aeb349cf1 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 14:55:43.843488 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1966159-244f-4e7c-97bb-971833a01b4a None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:43.844056 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1966159-244f-4e7c-97bb-971833a01b4a None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:43.844056 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1966159-244f-4e7c-97bb-971833a01b4a None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:43.845490 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 537/1085] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:43.850463 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1eb6e-09f9-4129-848e-2ec39889ef6b None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:43.850870 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1eb6e-09f9-4129-848e-2ec39889ef6b None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:43.851156 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1eb6e-09f9-4129-848e-2ec39889ef6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:43.869892 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-7dc1eb6e-09f9-4129-848e-2ec39889ef6b None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:43.871100 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7dc1eb6e-09f9-4129-848e-2ec39889ef6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:43.937552 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 549/1086] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:44.210308 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1966159-244f-4e7c-97bb-971833a01b4a None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:44.311666 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1966159-244f-4e7c-97bb-971833a01b4a None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:44.346083 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8375f2b-969a-44b5-91ea-2b30dd8df035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:44.346702 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8375f2b-969a-44b5-91ea-2b30dd8df035 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:44.347004 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8375f2b-969a-44b5-91ea-2b30dd8df035 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:44.347273 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8375f2b-969a-44b5-91ea-2b30dd8df035 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:44.357120 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8375f2b-969a-44b5-91ea-2b30dd8df035 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:55:44.361822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-e8375f2b-969a-44b5-91ea-2b30dd8df035 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:55:44.361822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:44.361822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:44.361822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:44.361822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:44.361822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:44.361822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:44.361822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:44.361822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:44.361822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:44.361822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:44.361822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:44.361822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:55:44.361822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:55:44.361822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:55:44.361822 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:55:44.362927 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:55:44.362927 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:55:44.362927 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:55:44.362927 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:55:44.362927 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:44.362927 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:44.362927 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:44.362927 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:44.362927 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:44.362927 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:44.362927 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:44.362927 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:44.362927 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:44.362927 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:44.362927 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:44.362927 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:44.363821 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:44.363821 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:44.363821 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:44.363821 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:44.363821 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:55:44.363821 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:55:44.363821 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:55:44.363821 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:55:44.363821 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:55:44.363821 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:55:44.363821 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:55:44.363821 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:44.363821 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 538/1087] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:55:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:44.368863 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1eb6e-09f9-4129-848e-2ec39889ef6b None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:44.386382 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:44.469431 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1eb6e-09f9-4129-848e-2ec39889ef6b None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:44.504633 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed5a45-30b3-4dfa-be0f-41ff9b687347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:44.505239 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed5a45-30b3-4dfa-be0f-41ff9b687347 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:44.505520 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed5a45-30b3-4dfa-be0f-41ff9b687347 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:44.505794 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed5a45-30b3-4dfa-be0f-41ff9b687347 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:44.506454 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed5a45-30b3-4dfa-be0f-41ff9b687347 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:44.506754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed5a45-30b3-4dfa-be0f-41ff9b687347 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:44.508157 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed5a45-30b3-4dfa-be0f-41ff9b687347 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:44.516773 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 550/1088] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:55:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:44.522691 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8375f2b-969a-44b5-91ea-2b30dd8df035 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:44.578930 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:44.623391 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8375f2b-969a-44b5-91ea-2b30dd8df035 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:44.656861 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24792690-755f-40bc-a9c8-dea6b2b4cf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:44.657496 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24792690-755f-40bc-a9c8-dea6b2b4cf79 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:44.657777 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24792690-755f-40bc-a9c8-dea6b2b4cf79 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:44.658053 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24792690-755f-40bc-a9c8-dea6b2b4cf79 None admin] PATH_INFO: `/v3/users/44f728866dc1443082b86698c151242d` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:44.659214 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24792690-755f-40bc-a9c8-dea6b2b4cf79 None admin] RBAC: Authorizing `identity:get_user(user_id=44f728866dc1443082b86698c151242d)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:44.660657 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24792690-755f-40bc-a9c8-dea6b2b4cf79 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:44.662242 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 539/1089] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:55:44 2022] GET /identity/v3/users/44f728866dc1443082b86698c151242d => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:55:44.667994 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed5a45-30b3-4dfa-be0f-41ff9b687347 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:44.747658 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:44.769564 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed5a45-30b3-4dfa-be0f-41ff9b687347 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:44.804504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cfcff6-e7db-4e76-9122-c95d66f18a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:44.805142 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cfcff6-e7db-4e76-9122-c95d66f18a18 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:44.805423 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cfcff6-e7db-4e76-9122-c95d66f18a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:44.805699 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cfcff6-e7db-4e76-9122-c95d66f18a18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:44.876046 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cfcff6-e7db-4e76-9122-c95d66f18a18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:44.877516 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cfcff6-e7db-4e76-9122-c95d66f18a18 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:44.911955 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d71c2-ef0f-4bb5-b0cc-56a75bc98d94 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:44.912474 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d71c2-ef0f-4bb5-b0cc-56a75bc98d94 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:44.912474 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d71c2-ef0f-4bb5-b0cc-56a75bc98d94 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:44.913820 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 540/1090] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:55:44.921063 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8107312-b45b-42a2-aaae-11020e38ab96 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:44.921347 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8107312-b45b-42a2-aaae-11020e38ab96 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:44.921623 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8107312-b45b-42a2-aaae-11020e38ab96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:44.940343 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-c8107312-b45b-42a2-aaae-11020e38ab96 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:44.941543 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-c8107312-b45b-42a2-aaae-11020e38ab96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:44.955076 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:44.978045 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 551/1091] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:55:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 14:55:44.984219 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cfcff6-e7db-4e76-9122-c95d66f18a18 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:45.010660 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 541/1092] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 14:55:45.089054 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cfcff6-e7db-4e76-9122-c95d66f18a18 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:45.121838 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e54c8-ac7c-4d69-9761-ada3ee97e1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:45.123042 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e54c8-ac7c-4d69-9761-ada3ee97e1ad None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:45.123321 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e54c8-ac7c-4d69-9761-ada3ee97e1ad None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:45.123595 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e54c8-ac7c-4d69-9761-ada3ee97e1ad None admin] PATH_INFO: `/v3/projects/829f2ddf22294cdd929f2edf944acfe8` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:45.124716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211e54c8-ac7c-4d69-9761-ada3ee97e1ad None admin] RBAC: Authorizing `identity:get_project(project_id=829f2ddf22294cdd929f2edf944acfe8)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:45.126158 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211e54c8-ac7c-4d69-9761-ada3ee97e1ad None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:45.127813 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 552/1093] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:55:44 2022] GET /identity/v3/projects/829f2ddf22294cdd929f2edf944acfe8 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:55:45.133836 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8107312-b45b-42a2-aaae-11020e38ab96 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:45.186561 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:45.232732 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8107312-b45b-42a2-aaae-11020e38ab96 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:45.273315 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a37d48f-885a-4906-87cc-89383f41db91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:45.273854 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37d48f-885a-4906-87cc-89383f41db91 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:45.274136 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37d48f-885a-4906-87cc-89383f41db91 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:45.274412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37d48f-885a-4906-87cc-89383f41db91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:45.275170 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a37d48f-885a-4906-87cc-89383f41db91 None admin] RBAC: Adding query filter params (role.id=99e8fe7df8e7446680a81d4eaaa1cb68, scope.project.id=829f2ddf22294cdd929f2edf944acfe8, user.id=44f728866dc1443082b86698c151242d) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:45.275479 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a37d48f-885a-4906-87cc-89383f41db91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:45.276909 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a37d48f-885a-4906-87cc-89383f41db91 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:45.282197 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e54c8-ac7c-4d69-9761-ada3ee97e1ad None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:45.284828 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 542/1094] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:55:45 2022] GET /identity/v3/role_assignments?user.id=44f728866dc1443082b86698c151242d&scope.project.id=829f2ddf22294cdd929f2edf944acfe8&role.id=99e8fe7df8e7446680a81d4eaaa1cb68 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:45.342144 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:45.398781 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e54c8-ac7c-4d69-9761-ada3ee97e1ad None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:45.436548 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b558a15-f67e-43f4-aefc-c35f92575b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:45.437193 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b558a15-f67e-43f4-aefc-c35f92575b28 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:45.437475 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b558a15-f67e-43f4-aefc-c35f92575b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:45.437748 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b558a15-f67e-43f4-aefc-c35f92575b28 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:45.448610 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b558a15-f67e-43f4-aefc-c35f92575b28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:55:45.453731 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-7b558a15-f67e-43f4-aefc-c35f92575b28 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 14:55:45.453731 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:45.453731 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:45.453731 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:45.453731 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:45.453731 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:45.453731 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:45.453731 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:45.453731 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:45.453731 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:45.453731 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:45.453731 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:45.453731 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:55:45.453731 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:55:45.453731 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:55:45.453731 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:55:45.454717 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:55:45.454717 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:55:45.454717 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:55:45.454717 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:55:45.454717 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:45.454717 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:45.454717 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:45.454717 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:45.454717 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:45.454717 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:45.454717 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:45.454717 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:45.454717 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:45.454717 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:45.454717 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:45.454717 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:45.459864 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:45.459864 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:45.459864 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:45.459864 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:45.459864 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:55:45.459864 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:55:45.459864 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:55:45.459864 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:55:45.459864 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:55:45.459864 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:55:45.459864 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 14:55:45.459864 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:45.459864 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 553/1095] 10.222.0.33 () {56 vars in 1120 bytes} [Sun Sep 11 14:55:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:55:45.463116 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a37d48f-885a-4906-87cc-89383f41db91 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:45.569496 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a37d48f-885a-4906-87cc-89383f41db91 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:45.585570 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:45.603987 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d1e7d-c4bb-4182-bac5-9da137ce0d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:45.604601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d1e7d-c4bb-4182-bac5-9da137ce0d3f None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:45.604884 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d1e7d-c4bb-4182-bac5-9da137ce0d3f None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:45.605161 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d1e7d-c4bb-4182-bac5-9da137ce0d3f None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:45.605821 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610d1e7d-c4bb-4182-bac5-9da137ce0d3f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:45.606124 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610d1e7d-c4bb-4182-bac5-9da137ce0d3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:45.608066 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610d1e7d-c4bb-4182-bac5-9da137ce0d3f None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:45.615940 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 543/1096] 10.222.0.33 () {56 vars in 1115 bytes} [Sun Sep 11 14:55:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:55:45.621699 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b558a15-f67e-43f4-aefc-c35f92575b28 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:45.721764 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b558a15-f67e-43f4-aefc-c35f92575b28 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:45.729636 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:45.757711 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e88f9a-8a5b-42ec-af78-08bbebf8f906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:45.758339 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e88f9a-8a5b-42ec-af78-08bbebf8f906 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:45.758620 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e88f9a-8a5b-42ec-af78-08bbebf8f906 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:45.758927 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e88f9a-8a5b-42ec-af78-08bbebf8f906 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:45.768843 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e88f9a-8a5b-42ec-af78-08bbebf8f906 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:45.770305 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e88f9a-8a5b-42ec-af78-08bbebf8f906 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:45.785553 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-b2e88f9a-8a5b-42ec-af78-08bbebf8f906 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:45.785553 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:45.785553 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:45.785553 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:45.785553 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:45.785553 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:45.785553 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:45.785553 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:45.785553 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:45.785553 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:45.785553 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:45.785553 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:45.785553 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:55:45.785553 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:55:45.785553 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:55:45.785553 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:55:45.786668 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:45.786668 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:45.786668 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:45.786668 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:45.786668 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:45.786668 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:45.786668 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:45.786668 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:45.786668 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:45.786668 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:45.786668 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:45.786668 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:45.786668 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:45.786668 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:45.786668 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:45.786668 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:45.786668 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:55:45.787805 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:55:45.787805 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:45.787805 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:45.790949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 554/1097] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:55:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:45.792432 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d1e7d-c4bb-4182-bac5-9da137ce0d3f None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:45.893675 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d1e7d-c4bb-4182-bac5-9da137ce0d3f None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:45.927288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67465a79-aa06-470f-b692-69814deda7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:45.927900 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67465a79-aa06-470f-b692-69814deda7ee None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:45.928153 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67465a79-aa06-470f-b692-69814deda7ee None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:45.928422 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67465a79-aa06-470f-b692-69814deda7ee None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:45.929049 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67465a79-aa06-470f-b692-69814deda7ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:45.929345 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67465a79-aa06-470f-b692-69814deda7ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:45.930704 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67465a79-aa06-470f-b692-69814deda7ee None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:45.942526 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 544/1098] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:55:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:45.948214 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e88f9a-8a5b-42ec-af78-08bbebf8f906 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:46.050996 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e88f9a-8a5b-42ec-af78-08bbebf8f906 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:46.084335 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f774819c-09bb-4c2d-8ff1-490c91de6b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:46.084960 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774819c-09bb-4c2d-8ff1-490c91de6b32 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:46.085240 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774819c-09bb-4c2d-8ff1-490c91de6b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:46.085515 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774819c-09bb-4c2d-8ff1-490c91de6b32 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:46.093465 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f774819c-09bb-4c2d-8ff1-490c91de6b32 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:46.094951 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f774819c-09bb-4c2d-8ff1-490c91de6b32 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:46.105917 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-f774819c-09bb-4c2d-8ff1-490c91de6b32 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 14:55:46.105917 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:46.105917 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:46.105917 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:46.105917 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:46.105917 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:46.105917 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:46.105917 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:46.105917 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:46.105917 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:46.105917 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:46.105917 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:46.105917 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:55:46.105917 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:55:46.105917 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:55:46.105917 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:55:46.106946 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:46.106946 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:46.106946 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:55:46.106946 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:55:46.106946 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:55:46.106946 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:55:46.106946 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:46.106946 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:46.106946 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:46.106946 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:46.106946 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:46.106946 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:46.106946 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:46.106946 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:46.106946 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:46.106946 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:46.106946 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:46.106946 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:46.108424 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:46.108424 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:46.108424 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:55:46.108424 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:55:46.108424 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:55:46.108424 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:55:46.108424 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:55:46.108424 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:55:46.108424 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 14:55:46.108424 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:55:46.108424 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 555/1099] 10.222.0.33 () {56 vars in 1093 bytes} [Sun Sep 11 14:55:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:55:46.113552 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67465a79-aa06-470f-b692-69814deda7ee None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:46.153809 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:46.217952 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67465a79-aa06-470f-b692-69814deda7ee None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:46.251829 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e1d47-9bca-4a7e-9de9-44d87ae8e84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:46.252432 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e1d47-9bca-4a7e-9de9-44d87ae8e84e None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:46.252718 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e1d47-9bca-4a7e-9de9-44d87ae8e84e None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:46.252990 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e1d47-9bca-4a7e-9de9-44d87ae8e84e None admin] PATH_INFO: `/v3/users` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:46.253706 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e1d47-9bca-4a7e-9de9-44d87ae8e84e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:46.254010 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e1d47-9bca-4a7e-9de9-44d87ae8e84e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:46.255482 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e1d47-9bca-4a7e-9de9-44d87ae8e84e None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:46.272999 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 545/1100] 10.222.0.33 () {56 vars in 1133 bytes} [Sun Sep 11 14:55:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:46.279420 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f774819c-09bb-4c2d-8ff1-490c91de6b32 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:46.298882 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:46.379736 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f774819c-09bb-4c2d-8ff1-490c91de6b32 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:46.413344 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c63bb7-bc18-4483-8eda-ec0c79cef821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:46.413884 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c63bb7-bc18-4483-8eda-ec0c79cef821 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:46.414165 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c63bb7-bc18-4483-8eda-ec0c79cef821 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:46.414437 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c63bb7-bc18-4483-8eda-ec0c79cef821 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:46.482214 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c63bb7-bc18-4483-8eda-ec0c79cef821 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:46.483698 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c63bb7-bc18-4483-8eda-ec0c79cef821 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:46.501197 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:46.583556 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 556/1101] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:55:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 14:55:46.589315 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e1d47-9bca-4a7e-9de9-44d87ae8e84e None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:46.681545 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:46.688687 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e1d47-9bca-4a7e-9de9-44d87ae8e84e None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:46.722521 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b469e55-dea4-4f8c-8f18-8ec5c4b186fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:46.723183 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b469e55-dea4-4f8c-8f18-8ec5c4b186fe None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:46.723466 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b469e55-dea4-4f8c-8f18-8ec5c4b186fe None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:46.723740 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b469e55-dea4-4f8c-8f18-8ec5c4b186fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:46.734799 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b469e55-dea4-4f8c-8f18-8ec5c4b186fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:46.736287 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b469e55-dea4-4f8c-8f18-8ec5c4b186fe None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:46.750944 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-3b469e55-dea4-4f8c-8f18-8ec5c4b186fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:46.750944 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:46.750944 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:46.750944 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:46.750944 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:46.750944 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:46.750944 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:46.750944 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:46.750944 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:46.750944 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:46.750944 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:46.750944 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:46.750944 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:55:46.750944 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:55:46.750944 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:55:46.750944 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:55:46.751959 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:46.751959 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:46.751959 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:46.751959 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:46.751959 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:46.751959 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:46.751959 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:46.751959 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:46.751959 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:46.751959 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:46.751959 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:46.751959 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:46.751959 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:46.751959 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:46.751959 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:46.751959 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:46.751959 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:55:46.752917 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:55:46.752917 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:55:46.752917 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:46.752917 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 546/1102] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:55:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:46.757743 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c63bb7-bc18-4483-8eda-ec0c79cef821 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:46.855147 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c63bb7-bc18-4483-8eda-ec0c79cef821 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:46.872869 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:46.890401 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e217936c-9008-41bf-a558-02547a3a597d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:46.891029 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217936c-9008-41bf-a558-02547a3a597d None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:46.891309 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217936c-9008-41bf-a558-02547a3a597d None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:46.891581 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217936c-9008-41bf-a558-02547a3a597d None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:46.892212 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e217936c-9008-41bf-a558-02547a3a597d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:46.892512 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e217936c-9008-41bf-a558-02547a3a597d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:46.893876 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e217936c-9008-41bf-a558-02547a3a597d None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:46.905097 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 557/1103] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:55:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:46.910718 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b469e55-dea4-4f8c-8f18-8ec5c4b186fe None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:47.015682 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b469e55-dea4-4f8c-8f18-8ec5c4b186fe None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:47.049147 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b48f22f-324f-461d-ad32-84f1ff208aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:47.049773 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48f22f-324f-461d-ad32-84f1ff208aa1 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:47.050051 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48f22f-324f-461d-ad32-84f1ff208aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:47.050323 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48f22f-324f-461d-ad32-84f1ff208aa1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:47.056827 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b48f22f-324f-461d-ad32-84f1ff208aa1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:47.058273 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b48f22f-324f-461d-ad32-84f1ff208aa1 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:47.070316 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-4b48f22f-324f-461d-ad32-84f1ff208aa1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:55:47.070316 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:55:47.070316 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:55:47.070316 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:55:47.070316 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:55:47.070316 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:55:47.070316 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:55:47.070316 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:55:47.070316 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:55:47.070316 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:55:47.070316 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:55:47.070316 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:55:47.070316 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:55:47.070316 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:55:47.070316 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:55:47.071311 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:55:47.071311 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:55:47.071311 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:55:47.071311 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:55:47.071311 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:55:47.071311 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:55:47.071311 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:55:47.071311 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:55:47.071311 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:55:47.071311 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:55:47.071311 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:55:47.071311 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:55:47.071311 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:55:47.071311 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:55:47.071311 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:55:47.071311 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:55:47.071311 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:55:47.072220 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:55:47.072220 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:55:47.072220 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:55:47.072220 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:55:47.072220 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:55:47.072220 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:55:47.072220 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:55:47.072220 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:55:47.072220 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 547/1104] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:55:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:55:47.077264 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e217936c-9008-41bf-a558-02547a3a597d None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:47.177073 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e217936c-9008-41bf-a558-02547a3a597d None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:47.210349 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93cc02c-ddb9-4265-9151-0a5850bad273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:47.210988 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93cc02c-ddb9-4265-9151-0a5850bad273 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:47.211267 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93cc02c-ddb9-4265-9151-0a5850bad273 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:47.211535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93cc02c-ddb9-4265-9151-0a5850bad273 None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:47.212219 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93cc02c-ddb9-4265-9151-0a5850bad273 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:47.212520 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93cc02c-ddb9-4265-9151-0a5850bad273 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:47.213957 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93cc02c-ddb9-4265-9151-0a5850bad273 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:47.225908 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 558/1105] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:55:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:47.232646 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b48f22f-324f-461d-ad32-84f1ff208aa1 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:47.244187 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:47.333529 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b48f22f-324f-461d-ad32-84f1ff208aa1 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:47.366773 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1043e6da-ec3d-452b-86d5-4c336f7acb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:47.367412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043e6da-ec3d-452b-86d5-4c336f7acb4d None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:47.367692 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043e6da-ec3d-452b-86d5-4c336f7acb4d None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:47.367971 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043e6da-ec3d-452b-86d5-4c336f7acb4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:47.368708 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1043e6da-ec3d-452b-86d5-4c336f7acb4d None admin] RBAC: Adding query filter params (role.id=56ed900824b74d938089a39aeb349cf1, scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176, user.id=fc0ea93084b0482fb2dd6d802987942f) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:47.369021 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1043e6da-ec3d-452b-86d5-4c336f7acb4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:47.370505 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1043e6da-ec3d-452b-86d5-4c336f7acb4d None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:47.377825 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 548/1106] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:55:47 2022] GET /identity/v3/role_assignments?user.id=fc0ea93084b0482fb2dd6d802987942f&scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176&role.id=56ed900824b74d938089a39aeb349cf1 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:47.505609 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d60aab-4801-4fd1-b4fc-3e5d1ead4707 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:47.505817 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d60aab-4801-4fd1-b4fc-3e5d1ead4707 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:47.506066 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d60aab-4801-4fd1-b4fc-3e5d1ead4707 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:47.507467 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 559/1107] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:47.513890 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877d766-1156-4c15-a4b9-1b44f2e51468 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:47.514248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877d766-1156-4c15-a4b9-1b44f2e51468 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:47.514528 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877d766-1156-4c15-a4b9-1b44f2e51468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:47.533918 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-f877d766-1156-4c15-a4b9-1b44f2e51468 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:47.535175 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-f877d766-1156-4c15-a4b9-1b44f2e51468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:47.605967 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 549/1108] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:47.877858 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d60aab-4801-4fd1-b4fc-3e5d1ead4707 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:47.922492 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:47.988244 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d60aab-4801-4fd1-b4fc-3e5d1ead4707 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:48.027703 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc54087-d79f-42a6-8c2d-0a6a4c650f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:48.028872 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc54087-d79f-42a6-8c2d-0a6a4c650f1c None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:48.029149 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc54087-d79f-42a6-8c2d-0a6a4c650f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:48.029418 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc54087-d79f-42a6-8c2d-0a6a4c650f1c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:48.030527 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc54087-d79f-42a6-8c2d-0a6a4c650f1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:48.032000 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc54087-d79f-42a6-8c2d-0a6a4c650f1c None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:48.033569 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 560/1109] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:55:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:48.039593 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f877d766-1156-4c15-a4b9-1b44f2e51468 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:48.041815 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:48.144131 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f877d766-1156-4c15-a4b9-1b44f2e51468 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:48.178241 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2523a2f6-fd95-4af0-b73c-3be044ab4419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:48.178870 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2523a2f6-fd95-4af0-b73c-3be044ab4419 None admin] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:48.179153 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2523a2f6-fd95-4af0-b73c-3be044ab4419 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:48.179429 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2523a2f6-fd95-4af0-b73c-3be044ab4419 None admin] PATH_INFO: `/v3/users` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:48.180086 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2523a2f6-fd95-4af0-b73c-3be044ab4419 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:48.181530 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2523a2f6-fd95-4af0-b73c-3be044ab4419 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:48.204973 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 550/1110] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 14:55:48 2022] POST /identity/v3/users => generated 286 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 14:55:50.391800 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8cea3b-2830-4ff9-bac4-ccc4e1a609f7 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:50.392313 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8cea3b-2830-4ff9-bac4-ccc4e1a609f7 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:50.392313 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8cea3b-2830-4ff9-bac4-ccc4e1a609f7 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:50.393605 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 561/1111] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:50.398931 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777e8e3-6836-4610-9ece-ad65a382d17a None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:50.399376 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777e8e3-6836-4610-9ece-ad65a382d17a None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:50.399479 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777e8e3-6836-4610-9ece-ad65a382d17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:50.418025 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-8777e8e3-6836-4610-9ece-ad65a382d17a None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:50.420180 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-8777e8e3-6836-4610-9ece-ad65a382d17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:50.471403 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:50.486184 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 551/1112] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:50.761498 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8cea3b-2830-4ff9-bac4-ccc4e1a609f7 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:50.862811 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8cea3b-2830-4ff9-bac4-ccc4e1a609f7 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:50.869863 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:50.897255 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cdbdc1-cf14-4cb9-9b70-00ef047eaa6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:50.897884 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cdbdc1-cf14-4cb9-9b70-00ef047eaa6e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:50.898166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cdbdc1-cf14-4cb9-9b70-00ef047eaa6e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:50.898440 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cdbdc1-cf14-4cb9-9b70-00ef047eaa6e None admin] PATH_INFO: `/v3/roles/d1818eb7f2bd467fbf75bad5cc8e0aea` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:50.899836 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cdbdc1-cf14-4cb9-9b70-00ef047eaa6e None admin] RBAC: Authorizing `identity:get_role(role_id=d1818eb7f2bd467fbf75bad5cc8e0aea)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:50.901209 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cdbdc1-cf14-4cb9-9b70-00ef047eaa6e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:50.902523 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 562/1113] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:55:50 2022] GET /identity/v3/roles/d1818eb7f2bd467fbf75bad5cc8e0aea => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:50.908873 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777e8e3-6836-4610-9ece-ad65a382d17a None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:51.004812 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777e8e3-6836-4610-9ece-ad65a382d17a None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:51.038356 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3eb089-21c3-4f05-86b3-d6f4a8fd7fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:51.038954 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3eb089-21c3-4f05-86b3-d6f4a8fd7fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:51.039239 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3eb089-21c3-4f05-86b3-d6f4a8fd7fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:51.039515 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3eb089-21c3-4f05-86b3-d6f4a8fd7fe3 None admin] PATH_INFO: `/v3/users/8c80f0c6f48644328a0d4c53b693e9d4` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:51.052324 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3eb089-21c3-4f05-86b3-d6f4a8fd7fe3 None admin] RBAC: Authorizing `identity:get_user(user_id=8c80f0c6f48644328a0d4c53b693e9d4)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:51.053760 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3eb089-21c3-4f05-86b3-d6f4a8fd7fe3 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:51.055379 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 552/1114] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:55:50 2022] GET /identity/v3/users/8c80f0c6f48644328a0d4c53b693e9d4 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:51.061957 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cdbdc1-cf14-4cb9-9b70-00ef047eaa6e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:51.158243 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cdbdc1-cf14-4cb9-9b70-00ef047eaa6e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:51.191465 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ade20-ec99-46e0-8fa6-897a57b02c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:51.192056 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ade20-ec99-46e0-8fa6-897a57b02c7f None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:51.192333 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ade20-ec99-46e0-8fa6-897a57b02c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:51.192602 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ade20-ec99-46e0-8fa6-897a57b02c7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:51.233125 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:51.257942 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976ade20-ec99-46e0-8fa6-897a57b02c7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:51.259421 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976ade20-ec99-46e0-8fa6-897a57b02c7f None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:51.354692 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 563/1115] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:55:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 14:55:51.360981 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3eb089-21c3-4f05-86b3-d6f4a8fd7fe3 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:51.407337 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:51.459926 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3eb089-21c3-4f05-86b3-d6f4a8fd7fe3 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:51.492939 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c19e9-6376-4ec1-8d45-2ac711d07309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:51.493551 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c19e9-6376-4ec1-8d45-2ac711d07309 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:51.493825 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c19e9-6376-4ec1-8d45-2ac711d07309 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:51.494089 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c19e9-6376-4ec1-8d45-2ac711d07309 None admin] PATH_INFO: `/v3/projects/829f2ddf22294cdd929f2edf944acfe8` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:51.495211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96c19e9-6376-4ec1-8d45-2ac711d07309 None admin] RBAC: Authorizing `identity:get_project(project_id=829f2ddf22294cdd929f2edf944acfe8)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:51.496634 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96c19e9-6376-4ec1-8d45-2ac711d07309 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:51.499147 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 553/1116] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:55:51 2022] GET /identity/v3/projects/829f2ddf22294cdd929f2edf944acfe8 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:51.505406 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ade20-ec99-46e0-8fa6-897a57b02c7f None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:51.579540 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:51.603732 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ade20-ec99-46e0-8fa6-897a57b02c7f None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:51.637386 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64edcd-23df-474b-acbc-d0a189e721e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:51.638117 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64edcd-23df-474b-acbc-d0a189e721e0 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:51.638403 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64edcd-23df-474b-acbc-d0a189e721e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:51.638673 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64edcd-23df-474b-acbc-d0a189e721e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:51.639415 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb64edcd-23df-474b-acbc-d0a189e721e0 None admin] RBAC: Adding query filter params (role.id=d1818eb7f2bd467fbf75bad5cc8e0aea, scope.project.id=829f2ddf22294cdd929f2edf944acfe8, user.id=8c80f0c6f48644328a0d4c53b693e9d4) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:51.639720 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb64edcd-23df-474b-acbc-d0a189e721e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:51.641149 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb64edcd-23df-474b-acbc-d0a189e721e0 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:51.646415 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 564/1117] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:55:51 2022] GET /identity/v3/role_assignments?user.id=8c80f0c6f48644328a0d4c53b693e9d4&scope.project.id=829f2ddf22294cdd929f2edf944acfe8&role.id=d1818eb7f2bd467fbf75bad5cc8e0aea => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:53.763662 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3eace2-d93e-42e1-baaa-a7f73503b4c2 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:53.764198 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3eace2-d93e-42e1-baaa-a7f73503b4c2 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:53.764198 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3eace2-d93e-42e1-baaa-a7f73503b4c2 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:53.765538 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 554/1118] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:53.770679 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4ad16-3518-4803-93f3-c0f991e2c0f2 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:53.771134 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4ad16-3518-4803-93f3-c0f991e2c0f2 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:53.771261 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4ad16-3518-4803-93f3-c0f991e2c0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:53.789506 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-36d4ad16-3518-4803-93f3-c0f991e2c0f2 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:53.790678 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-36d4ad16-3518-4803-93f3-c0f991e2c0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:53.855641 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 565/1119] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:54.128939 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3eace2-d93e-42e1-baaa-a7f73503b4c2 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:54.227202 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3eace2-d93e-42e1-baaa-a7f73503b4c2 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:54.240507 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:54.260874 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55d221-d0fe-4b96-a416-63f08fa55651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:54.261485 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55d221-d0fe-4b96-a416-63f08fa55651 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:54.261760 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55d221-d0fe-4b96-a416-63f08fa55651 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:54.262028 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55d221-d0fe-4b96-a416-63f08fa55651 None admin] PATH_INFO: `/v3/roles/d1818eb7f2bd467fbf75bad5cc8e0aea` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:54.264050 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd55d221-d0fe-4b96-a416-63f08fa55651 None admin] RBAC: Authorizing `identity:get_role(role_id=d1818eb7f2bd467fbf75bad5cc8e0aea)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:54.265410 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd55d221-d0fe-4b96-a416-63f08fa55651 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:54.266702 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 555/1120] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:55:54 2022] GET /identity/v3/roles/d1818eb7f2bd467fbf75bad5cc8e0aea => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:54.272456 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d4ad16-3518-4803-93f3-c0f991e2c0f2 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:54.337337 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:54.369295 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d4ad16-3518-4803-93f3-c0f991e2c0f2 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:54.402689 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483960cc-a058-4515-af7e-c5beacbd62b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:54.403480 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483960cc-a058-4515-af7e-c5beacbd62b5 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:54.403970 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483960cc-a058-4515-af7e-c5beacbd62b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:54.404251 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483960cc-a058-4515-af7e-c5beacbd62b5 None admin] PATH_INFO: `/v3/users/8c80f0c6f48644328a0d4c53b693e9d4` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:54.405361 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483960cc-a058-4515-af7e-c5beacbd62b5 None admin] RBAC: Authorizing `identity:get_user(user_id=8c80f0c6f48644328a0d4c53b693e9d4)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:54.406788 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483960cc-a058-4515-af7e-c5beacbd62b5 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:54.408379 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 566/1121] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:55:54 2022] GET /identity/v3/users/8c80f0c6f48644328a0d4c53b693e9d4 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:54.414078 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55d221-d0fe-4b96-a416-63f08fa55651 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:54.512253 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55d221-d0fe-4b96-a416-63f08fa55651 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:54.545384 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4ed04-fa79-442c-8f31-58c0bac28dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:54.546003 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4ed04-fa79-442c-8f31-58c0bac28dab None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:54.546284 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4ed04-fa79-442c-8f31-58c0bac28dab None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:54.546557 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4ed04-fa79-442c-8f31-58c0bac28dab None admin] PATH_INFO: `/v3/projects/829f2ddf22294cdd929f2edf944acfe8` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:54.547697 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f4ed04-fa79-442c-8f31-58c0bac28dab None admin] RBAC: Authorizing `identity:get_project(project_id=829f2ddf22294cdd929f2edf944acfe8)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:54.549129 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f4ed04-fa79-442c-8f31-58c0bac28dab None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:54.550734 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 556/1122] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:55:54 2022] GET /identity/v3/projects/829f2ddf22294cdd929f2edf944acfe8 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:54.556585 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483960cc-a058-4515-af7e-c5beacbd62b5 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:54.653849 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483960cc-a058-4515-af7e-c5beacbd62b5 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:54.687971 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb23bbe-2c13-44a6-a22a-2e21832a4675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:54.688615 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb23bbe-2c13-44a6-a22a-2e21832a4675 None admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:54.688895 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb23bbe-2c13-44a6-a22a-2e21832a4675 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:54.689168 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb23bbe-2c13-44a6-a22a-2e21832a4675 None admin] PATH_INFO: `/v3/projects/829f2ddf22294cdd929f2edf944acfe8/users/8c80f0c6f48644328a0d4c53b693e9d4/roles/d1818eb7f2bd467fbf75bad5cc8e0aea` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:54.691329 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb23bbe-2c13-44a6-a22a-2e21832a4675 None admin] RBAC: Authorizing `identity:create_grant(project_id=829f2ddf22294cdd929f2edf944acfe8, user_id=8c80f0c6f48644328a0d4c53b693e9d4, role_id=d1818eb7f2bd467fbf75bad5cc8e0aea)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:54.692762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb23bbe-2c13-44a6-a22a-2e21832a4675 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:54.695720 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:54.706361 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 567/1123] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 14:55:54 2022] PUT /identity/v3/projects/829f2ddf22294cdd929f2edf944acfe8/users/8c80f0c6f48644328a0d4c53b693e9d4/roles/d1818eb7f2bd467fbf75bad5cc8e0aea => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:55:56.814906 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8445a3-cb16-4a17-9158-075631a19e28 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:56.815442 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8445a3-cb16-4a17-9158-075631a19e28 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:56.815442 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8445a3-cb16-4a17-9158-075631a19e28 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:56.816776 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 557/1124] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:55:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:55:56.821920 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c4d55-b48b-457b-9025-f74d97ee920d None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:56.822319 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c4d55-b48b-457b-9025-f74d97ee920d None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:56.822473 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c4d55-b48b-457b-9025-f74d97ee920d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:56.840559 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-977c4d55-b48b-457b-9025-f74d97ee920d None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:55:56.841758 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-977c4d55-b48b-457b-9025-f74d97ee920d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:55:56.907371 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 568/1125] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:55:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:55:57.176759 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8445a3-cb16-4a17-9158-075631a19e28 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:57.227316 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:57.276854 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8445a3-cb16-4a17-9158-075631a19e28 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:57.310826 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d775c23-4485-4bdf-90e7-eacd7d17a47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:57.311463 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d775c23-4485-4bdf-90e7-eacd7d17a47b None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:57.311743 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d775c23-4485-4bdf-90e7-eacd7d17a47b None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:57.312015 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d775c23-4485-4bdf-90e7-eacd7d17a47b None admin] PATH_INFO: `/v3/roles/d1818eb7f2bd467fbf75bad5cc8e0aea` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:57.313394 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d775c23-4485-4bdf-90e7-eacd7d17a47b None admin] RBAC: Authorizing `identity:get_role(role_id=d1818eb7f2bd467fbf75bad5cc8e0aea)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:57.314763 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d775c23-4485-4bdf-90e7-eacd7d17a47b None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:57.316116 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 558/1126] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:55:57 2022] GET /identity/v3/roles/d1818eb7f2bd467fbf75bad5cc8e0aea => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:57.324531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c4d55-b48b-457b-9025-f74d97ee920d None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:57.421881 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c4d55-b48b-457b-9025-f74d97ee920d None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:57.446858 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:57.456263 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3cc5c3-a739-4d71-8e5e-3b20468a4210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:57.456892 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3cc5c3-a739-4d71-8e5e-3b20468a4210 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:57.457175 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3cc5c3-a739-4d71-8e5e-3b20468a4210 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:57.457448 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3cc5c3-a739-4d71-8e5e-3b20468a4210 None admin] PATH_INFO: `/v3/users/8c80f0c6f48644328a0d4c53b693e9d4` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:57.458566 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3cc5c3-a739-4d71-8e5e-3b20468a4210 None admin] RBAC: Authorizing `identity:get_user(user_id=8c80f0c6f48644328a0d4c53b693e9d4)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:57.460050 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3cc5c3-a739-4d71-8e5e-3b20468a4210 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:57.461618 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 569/1127] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:55:57 2022] GET /identity/v3/users/8c80f0c6f48644328a0d4c53b693e9d4 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:55:57.468026 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d775c23-4485-4bdf-90e7-eacd7d17a47b None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:57.565252 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d775c23-4485-4bdf-90e7-eacd7d17a47b None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:57.582355 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:57.599590 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a3bd9-0ed2-4fca-88a8-d9728df5e49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:57.600190 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a3bd9-0ed2-4fca-88a8-d9728df5e49f None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:57.600471 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a3bd9-0ed2-4fca-88a8-d9728df5e49f None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:57.600744 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a3bd9-0ed2-4fca-88a8-d9728df5e49f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:57.665682 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2a3bd9-0ed2-4fca-88a8-d9728df5e49f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:57.667158 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2a3bd9-0ed2-4fca-88a8-d9728df5e49f None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:57.762082 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 559/1128] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:55:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 14:55:57.768431 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3cc5c3-a739-4d71-8e5e-3b20468a4210 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:57.867656 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3cc5c3-a739-4d71-8e5e-3b20468a4210 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:57.901613 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27653e-b177-4a70-9306-7f1bd5340ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:57.902232 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27653e-b177-4a70-9306-7f1bd5340ccd None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:57.902510 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27653e-b177-4a70-9306-7f1bd5340ccd None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:57.902781 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27653e-b177-4a70-9306-7f1bd5340ccd None admin] PATH_INFO: `/v3/projects/829f2ddf22294cdd929f2edf944acfe8` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:57.903970 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b27653e-b177-4a70-9306-7f1bd5340ccd None admin] RBAC: Authorizing `identity:get_project(project_id=829f2ddf22294cdd929f2edf944acfe8)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:57.905406 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b27653e-b177-4a70-9306-7f1bd5340ccd None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:57.907054 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 570/1129] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:55:57 2022] GET /identity/v3/projects/829f2ddf22294cdd929f2edf944acfe8 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:55:57.913213 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a3bd9-0ed2-4fca-88a8-d9728df5e49f None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:55:57.931133 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:55:58.013841 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a3bd9-0ed2-4fca-88a8-d9728df5e49f None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:55:58.046559 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc863d-7a35-43f8-903d-9540d6eaef73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:55:58.047189 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc863d-7a35-43f8-903d-9540d6eaef73 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:55:58.047470 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc863d-7a35-43f8-903d-9540d6eaef73 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:55:58.047743 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc863d-7a35-43f8-903d-9540d6eaef73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:55:58.048461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc863d-7a35-43f8-903d-9540d6eaef73 None admin] RBAC: Adding query filter params (role.id=d1818eb7f2bd467fbf75bad5cc8e0aea, scope.project.id=829f2ddf22294cdd929f2edf944acfe8, user.id=8c80f0c6f48644328a0d4c53b693e9d4) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:55:58.048770 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc863d-7a35-43f8-903d-9540d6eaef73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:55:58.050204 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc863d-7a35-43f8-903d-9540d6eaef73 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:55:58.055830 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 560/1130] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:55:57 2022] GET /identity/v3/role_assignments?user.id=8c80f0c6f48644328a0d4c53b693e9d4&scope.project.id=829f2ddf22294cdd929f2edf944acfe8&role.id=d1818eb7f2bd467fbf75bad5cc8e0aea => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:56:00.212231 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617aec4d-4887-4fc3-8379-a2edd5169233 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:00.212791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617aec4d-4887-4fc3-8379-a2edd5169233 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:00.212791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617aec4d-4887-4fc3-8379-a2edd5169233 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:00.214107 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 571/1131] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:56:00.219863 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f22643-40b3-4753-99ac-02a1e239d008 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:00.220249 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f22643-40b3-4753-99ac-02a1e239d008 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:00.220439 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f22643-40b3-4753-99ac-02a1e239d008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:00.238878 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-e5f22643-40b3-4753-99ac-02a1e239d008 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:56:00.240106 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-e5f22643-40b3-4753-99ac-02a1e239d008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:56:00.306491 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 561/1132] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:56:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:56:00.580966 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617aec4d-4887-4fc3-8379-a2edd5169233 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:00.642510 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:00.681725 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617aec4d-4887-4fc3-8379-a2edd5169233 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:00.715074 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506cdef6-dadf-40de-aeb2-7964919abbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:00.716257 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cdef6-dadf-40de-aeb2-7964919abbac None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:00.716621 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cdef6-dadf-40de-aeb2-7964919abbac None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:00.716896 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cdef6-dadf-40de-aeb2-7964919abbac None admin] PATH_INFO: `/v3/domains/default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:00.718014 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506cdef6-dadf-40de-aeb2-7964919abbac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:00.719488 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506cdef6-dadf-40de-aeb2-7964919abbac None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:00.721046 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 572/1133] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:56:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:56:00.726833 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f22643-40b3-4753-99ac-02a1e239d008 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:00.740654 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:00.824465 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f22643-40b3-4753-99ac-02a1e239d008 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:00.858530 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a692590-a2dd-4b39-96e5-1de35b5f89f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:00.859103 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a692590-a2dd-4b39-96e5-1de35b5f89f0 None admin] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:00.859385 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a692590-a2dd-4b39-96e5-1de35b5f89f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:00.859662 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a692590-a2dd-4b39-96e5-1de35b5f89f0 None admin] PATH_INFO: `/v3/projects` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:00.860316 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a692590-a2dd-4b39-96e5-1de35b5f89f0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:00.861756 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a692590-a2dd-4b39-96e5-1de35b5f89f0 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:00.888480 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 562/1134] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 11 14:56:00 2022] POST /identity/v3/projects => generated 311 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:56:03.069496 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6f4e7-2a1a-4ec4-937f-87b958848b9c None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:03.070018 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6f4e7-2a1a-4ec4-937f-87b958848b9c None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:03.070111 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6f4e7-2a1a-4ec4-937f-87b958848b9c None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:03.071415 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 573/1135] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:56:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:56:03.077051 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098bdc8-01a1-44dd-9527-e7bf6b5302ef None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:03.077569 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098bdc8-01a1-44dd-9527-e7bf6b5302ef None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:03.077569 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098bdc8-01a1-44dd-9527-e7bf6b5302ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:03.096093 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-d098bdc8-01a1-44dd-9527-e7bf6b5302ef None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:56:03.097276 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-d098bdc8-01a1-44dd-9527-e7bf6b5302ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:56:03.154265 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:03.163214 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 563/1136] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:56:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:56:03.442593 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c6f4e7-2a1a-4ec4-937f-87b958848b9c None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:03.560274 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c6f4e7-2a1a-4ec4-937f-87b958848b9c None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:03.599059 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:03.605693 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce19bde1-bc7d-4b74-b564-fee265d07e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:03.606387 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce19bde1-bc7d-4b74-b564-fee265d07e45 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:03.606666 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce19bde1-bc7d-4b74-b564-fee265d07e45 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:03.606975 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce19bde1-bc7d-4b74-b564-fee265d07e45 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:03.608102 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce19bde1-bc7d-4b74-b564-fee265d07e45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:03.609538 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce19bde1-bc7d-4b74-b564-fee265d07e45 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:03.611355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 574/1137] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:56:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:56:03.617145 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d098bdc8-01a1-44dd-9527-e7bf6b5302ef None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:03.729297 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d098bdc8-01a1-44dd-9527-e7bf6b5302ef None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:03.762115 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08862a0-4130-4343-8676-9f698bb32038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:03.763262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08862a0-4130-4343-8676-9f698bb32038 None admin] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:03.763540 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08862a0-4130-4343-8676-9f698bb32038 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:03.763809 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08862a0-4130-4343-8676-9f698bb32038 None admin] PATH_INFO: `/v3/users` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:03.764461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08862a0-4130-4343-8676-9f698bb32038 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:03.765898 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08862a0-4130-4343-8676-9f698bb32038 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:03.786631 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 564/1138] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 14:56:03 2022] POST /identity/v3/users => generated 286 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:56:05.956370 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e463a-6b92-46bc-ad44-8fa2a8fe8694 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:05.956909 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e463a-6b92-46bc-ad44-8fa2a8fe8694 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:05.956909 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e463a-6b92-46bc-ad44-8fa2a8fe8694 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:05.958255 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 575/1139] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:56:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:56:05.963410 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf7bcf-88b2-456a-a432-ff34d6c9697b None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:05.963817 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf7bcf-88b2-456a-a432-ff34d6c9697b None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:05.963960 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf7bcf-88b2-456a-a432-ff34d6c9697b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:05.982539 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-fcaf7bcf-88b2-456a-a432-ff34d6c9697b None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:56:05.983768 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-fcaf7bcf-88b2-456a-a432-ff34d6c9697b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:56:06.029469 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:06.051065 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 565/1140] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:56:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:56:06.321835 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e463a-6b92-46bc-ad44-8fa2a8fe8694 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:06.420840 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e463a-6b92-46bc-ad44-8fa2a8fe8694 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:06.453873 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e484e-ad95-4770-b812-308854d1109b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:06.454487 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e484e-ad95-4770-b812-308854d1109b None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:06.454765 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e484e-ad95-4770-b812-308854d1109b None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:06.455066 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e484e-ad95-4770-b812-308854d1109b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:06.464811 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e484e-ad95-4770-b812-308854d1109b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:56:06.469755 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-ec4e484e-ad95-4770-b812-308854d1109b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:56:06.469755 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:56:06.469755 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:56:06.469755 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:56:06.469755 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:56:06.469755 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:56:06.469755 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:56:06.469755 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:56:06.469755 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:56:06.469755 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:56:06.469755 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:56:06.469755 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:56:06.469755 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:56:06.469755 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:56:06.469755 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:56:06.469755 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:56:06.470867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:56:06.470867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:56:06.470867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:56:06.470867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:56:06.470867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:56:06.470867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:56:06.470867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:56:06.470867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:56:06.470867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:56:06.470867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:56:06.470867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:56:06.470867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:56:06.470867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:56:06.470867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:56:06.470867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:56:06.470867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:56:06.472069 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:56:06.472069 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:56:06.472069 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:56:06.472069 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:56:06.472069 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:56:06.472069 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:56:06.472069 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:56:06.472069 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:56:06.472069 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:56:06.472069 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:56:06.472069 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:56:06.472069 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:56:06.472069 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 576/1141] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:56:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:56:06.476782 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf7bcf-88b2-456a-a432-ff34d6c9697b None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:06.574530 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf7bcf-88b2-456a-a432-ff34d6c9697b None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:06.607356 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb6df1-e66f-475c-8c1e-70ca3015215a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:06.607962 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb6df1-e66f-475c-8c1e-70ca3015215a None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:06.608243 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb6df1-e66f-475c-8c1e-70ca3015215a None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:06.608516 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb6df1-e66f-475c-8c1e-70ca3015215a None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:06.609181 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb6df1-e66f-475c-8c1e-70ca3015215a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:56:06.609478 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb6df1-e66f-475c-8c1e-70ca3015215a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:06.610890 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb6df1-e66f-475c-8c1e-70ca3015215a None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:06.618770 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 566/1142] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:56:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:56:06.625418 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e484e-ad95-4770-b812-308854d1109b None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:06.682997 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:06.724081 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e484e-ad95-4770-b812-308854d1109b None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:06.756929 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623fed69-a05b-4a40-9594-624fd45fbdd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:06.757541 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fed69-a05b-4a40-9594-624fd45fbdd2 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:06.757816 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fed69-a05b-4a40-9594-624fd45fbdd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:06.758086 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fed69-a05b-4a40-9594-624fd45fbdd2 None admin] PATH_INFO: `/v3/users/e5441b73c4f94560873731648e44d1b9` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:06.770828 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623fed69-a05b-4a40-9594-624fd45fbdd2 None admin] RBAC: Authorizing `identity:get_user(user_id=e5441b73c4f94560873731648e44d1b9)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:06.772283 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623fed69-a05b-4a40-9594-624fd45fbdd2 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:06.773841 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 577/1143] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:56:06 2022] GET /identity/v3/users/e5441b73c4f94560873731648e44d1b9 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:56:06.780152 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb6df1-e66f-475c-8c1e-70ca3015215a None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:06.823511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:06.879608 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb6df1-e66f-475c-8c1e-70ca3015215a None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:06.912405 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf80237-3f18-4ba2-b31e-75941659d0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:06.912996 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf80237-3f18-4ba2-b31e-75941659d0ff None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:06.913270 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf80237-3f18-4ba2-b31e-75941659d0ff None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:06.913535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf80237-3f18-4ba2-b31e-75941659d0ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:06.978637 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf80237-3f18-4ba2-b31e-75941659d0ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:06.980118 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf80237-3f18-4ba2-b31e-75941659d0ff None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:07.020997 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:07.077681 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 567/1144] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:56:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 14:56:07.083670 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623fed69-a05b-4a40-9594-624fd45fbdd2 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:07.185715 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623fed69-a05b-4a40-9594-624fd45fbdd2 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:07.201898 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:07.220266 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63abede3-dbfd-4c3e-8a03-f3179acfb457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:07.220884 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abede3-dbfd-4c3e-8a03-f3179acfb457 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:07.221160 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abede3-dbfd-4c3e-8a03-f3179acfb457 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:07.221427 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abede3-dbfd-4c3e-8a03-f3179acfb457 None admin] PATH_INFO: `/v3/projects/75cdab48e60f473f9e4fc6e63b3d3a45` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:07.222612 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63abede3-dbfd-4c3e-8a03-f3179acfb457 None admin] RBAC: Authorizing `identity:get_project(project_id=75cdab48e60f473f9e4fc6e63b3d3a45)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:07.224076 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63abede3-dbfd-4c3e-8a03-f3179acfb457 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:07.225685 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 578/1145] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:56:07 2022] GET /identity/v3/projects/75cdab48e60f473f9e4fc6e63b3d3a45 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:56:07.232014 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf80237-3f18-4ba2-b31e-75941659d0ff None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:07.329179 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf80237-3f18-4ba2-b31e-75941659d0ff None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:07.362497 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d1797-ede1-4506-8c07-022de947d366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:07.363120 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d1797-ede1-4506-8c07-022de947d366 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:07.363402 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d1797-ede1-4506-8c07-022de947d366 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:07.363675 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d1797-ede1-4506-8c07-022de947d366 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:07.364399 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83d1797-ede1-4506-8c07-022de947d366 None admin] RBAC: Adding query filter params (role.id=99e8fe7df8e7446680a81d4eaaa1cb68, scope.project.id=75cdab48e60f473f9e4fc6e63b3d3a45, user.id=e5441b73c4f94560873731648e44d1b9) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:56:07.364705 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83d1797-ede1-4506-8c07-022de947d366 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:07.366135 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83d1797-ede1-4506-8c07-022de947d366 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:07.371405 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 568/1146] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:56:07 2022] GET /identity/v3/role_assignments?user.id=e5441b73c4f94560873731648e44d1b9&scope.project.id=75cdab48e60f473f9e4fc6e63b3d3a45&role.id=99e8fe7df8e7446680a81d4eaaa1cb68 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:56:09.613570 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346628df-7ec4-4206-9bb7-5cbe05a5e9b3 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:09.614143 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346628df-7ec4-4206-9bb7-5cbe05a5e9b3 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:09.614143 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346628df-7ec4-4206-9bb7-5cbe05a5e9b3 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:09.615455 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 579/1147] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:56:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:56:09.620525 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c407a3-fd37-4658-ace2-ea45b3ece067 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:09.620950 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c407a3-fd37-4658-ace2-ea45b3ece067 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:09.621061 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c407a3-fd37-4658-ace2-ea45b3ece067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:09.634371 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:09.640534 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-54c407a3-fd37-4658-ace2-ea45b3ece067 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:56:09.641731 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-54c407a3-fd37-4658-ace2-ea45b3ece067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:56:09.706875 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 569/1148] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:56:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:56:09.973229 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346628df-7ec4-4206-9bb7-5cbe05a5e9b3 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:10.073599 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346628df-7ec4-4206-9bb7-5cbe05a5e9b3 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:10.107009 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f74578-479d-4b76-b2a1-331374dd318e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:10.107618 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f74578-479d-4b76-b2a1-331374dd318e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:10.107893 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f74578-479d-4b76-b2a1-331374dd318e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:10.108159 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f74578-479d-4b76-b2a1-331374dd318e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:10.117836 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f74578-479d-4b76-b2a1-331374dd318e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:56:10.123226 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-75f74578-479d-4b76-b2a1-331374dd318e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:56:10.123226 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:56:10.123226 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:56:10.123226 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:56:10.123226 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:56:10.123226 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:56:10.123226 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:56:10.123226 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:56:10.123226 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:56:10.123226 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:56:10.123226 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:56:10.123226 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:56:10.123226 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:56:10.123226 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:56:10.123226 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:56:10.123226 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:56:10.124446 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:56:10.124446 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:56:10.124446 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:56:10.124446 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:56:10.124446 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:56:10.124446 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:56:10.124446 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:56:10.124446 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:56:10.124446 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:56:10.124446 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:56:10.124446 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:56:10.124446 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:56:10.124446 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:56:10.124446 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:56:10.124446 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:56:10.124446 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:56:10.125378 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:56:10.125378 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:56:10.125378 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:56:10.125378 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:56:10.125378 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:56:10.125378 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:56:10.125378 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:56:10.125378 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:56:10.125378 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:56:10.125378 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:56:10.125378 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:56:10.125378 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:56:10.125378 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 580/1149] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:56:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:56:10.130373 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c407a3-fd37-4658-ace2-ea45b3ece067 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:10.228083 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c407a3-fd37-4658-ace2-ea45b3ece067 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:10.260966 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb326f5-69f0-4116-9a77-4be898e9bbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:10.261558 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb326f5-69f0-4116-9a77-4be898e9bbc5 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:10.261834 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb326f5-69f0-4116-9a77-4be898e9bbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:10.262238 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb326f5-69f0-4116-9a77-4be898e9bbc5 None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:10.263168 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb326f5-69f0-4116-9a77-4be898e9bbc5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:56:10.263472 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb326f5-69f0-4116-9a77-4be898e9bbc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:10.264844 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb326f5-69f0-4116-9a77-4be898e9bbc5 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:10.266965 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:10.273222 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 570/1150] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:56:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:56:10.279544 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f74578-479d-4b76-b2a1-331374dd318e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:10.324698 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:10.377517 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f74578-479d-4b76-b2a1-331374dd318e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:10.410822 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d4e369-e554-4f52-9856-17552fe8a396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:10.411438 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4e369-e554-4f52-9856-17552fe8a396 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:10.411712 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4e369-e554-4f52-9856-17552fe8a396 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:10.411978 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4e369-e554-4f52-9856-17552fe8a396 None admin] PATH_INFO: `/v3/users/e5441b73c4f94560873731648e44d1b9` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:10.413127 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d4e369-e554-4f52-9856-17552fe8a396 None admin] RBAC: Authorizing `identity:get_user(user_id=e5441b73c4f94560873731648e44d1b9)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:10.414688 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d4e369-e554-4f52-9856-17552fe8a396 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:10.416260 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 581/1151] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:56:10 2022] GET /identity/v3/users/e5441b73c4f94560873731648e44d1b9 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:56:10.422612 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb326f5-69f0-4116-9a77-4be898e9bbc5 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:10.519012 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb326f5-69f0-4116-9a77-4be898e9bbc5 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:10.551689 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722ab36-b719-486d-b230-a5f3d3616ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:10.552305 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722ab36-b719-486d-b230-a5f3d3616ea2 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:10.552585 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722ab36-b719-486d-b230-a5f3d3616ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:10.552856 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722ab36-b719-486d-b230-a5f3d3616ea2 None admin] PATH_INFO: `/v3/projects/75cdab48e60f473f9e4fc6e63b3d3a45` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:10.553963 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a722ab36-b719-486d-b230-a5f3d3616ea2 None admin] RBAC: Authorizing `identity:get_project(project_id=75cdab48e60f473f9e4fc6e63b3d3a45)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:10.555421 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a722ab36-b719-486d-b230-a5f3d3616ea2 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:10.557028 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 571/1152] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:56:10 2022] GET /identity/v3/projects/75cdab48e60f473f9e4fc6e63b3d3a45 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:56:10.563136 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d4e369-e554-4f52-9856-17552fe8a396 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:10.659774 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d4e369-e554-4f52-9856-17552fe8a396 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:10.672619 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:10.694788 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494c8f0-7b44-4b7e-b783-fcbc626f7eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:10.695456 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494c8f0-7b44-4b7e-b783-fcbc626f7eca None admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:10.695733 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494c8f0-7b44-4b7e-b783-fcbc626f7eca None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:10.696005 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494c8f0-7b44-4b7e-b783-fcbc626f7eca None admin] PATH_INFO: `/v3/projects/75cdab48e60f473f9e4fc6e63b3d3a45/users/e5441b73c4f94560873731648e44d1b9/roles/99e8fe7df8e7446680a81d4eaaa1cb68` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:10.697878 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4494c8f0-7b44-4b7e-b783-fcbc626f7eca None admin] RBAC: Authorizing `identity:create_grant(project_id=75cdab48e60f473f9e4fc6e63b3d3a45, user_id=e5441b73c4f94560873731648e44d1b9, role_id=99e8fe7df8e7446680a81d4eaaa1cb68)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:10.699339 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4494c8f0-7b44-4b7e-b783-fcbc626f7eca None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:10.712966 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 582/1153] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 14:56:10 2022] PUT /identity/v3/projects/75cdab48e60f473f9e4fc6e63b3d3a45/users/e5441b73c4f94560873731648e44d1b9/roles/99e8fe7df8e7446680a81d4eaaa1cb68 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:56:12.845863 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d07f7-3bba-40fa-9ee0-58170c466b68 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:12.846426 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d07f7-3bba-40fa-9ee0-58170c466b68 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:12.846426 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d07f7-3bba-40fa-9ee0-58170c466b68 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:12.847774 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 572/1154] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:56:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:56:12.853128 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc7b22-7f6b-437f-b099-fe425d1c0deb None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:12.853534 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc7b22-7f6b-437f-b099-fe425d1c0deb None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:12.853654 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc7b22-7f6b-437f-b099-fe425d1c0deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:12.871297 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-b7dc7b22-7f6b-437f-b099-fe425d1c0deb None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:56:12.872753 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-b7dc7b22-7f6b-437f-b099-fe425d1c0deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:56:12.938196 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 583/1155] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:56:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:56:13.210191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1d07f7-3bba-40fa-9ee0-58170c466b68 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:13.276378 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:13.312843 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1d07f7-3bba-40fa-9ee0-58170c466b68 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:13.346030 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68e980-099c-4cf9-a912-cd0bd2e89373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:13.346656 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68e980-099c-4cf9-a912-cd0bd2e89373 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:13.346974 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68e980-099c-4cf9-a912-cd0bd2e89373 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:13.347250 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68e980-099c-4cf9-a912-cd0bd2e89373 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:13.358303 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68e980-099c-4cf9-a912-cd0bd2e89373 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:56:13.363141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-5f68e980-099c-4cf9-a912-cd0bd2e89373 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:56:13.363141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:56:13.363141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:56:13.363141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:56:13.363141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:56:13.363141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:56:13.363141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:56:13.363141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:56:13.363141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:56:13.363141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:56:13.363141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:56:13.363141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:56:13.363141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:56:13.363141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:56:13.363141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:56:13.363141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:56:13.364248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:56:13.364248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:56:13.364248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:56:13.364248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:56:13.364248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:56:13.364248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:56:13.364248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:56:13.364248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:56:13.364248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:56:13.364248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:56:13.364248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:56:13.364248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:56:13.364248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:56:13.364248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:56:13.364248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:56:13.364248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:56:13.365133 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:56:13.365133 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:56:13.365133 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:56:13.365133 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:56:13.365133 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:56:13.365133 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:56:13.365133 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:56:13.365133 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:56:13.365133 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:56:13.365133 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:56:13.365133 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:56:13.365133 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:56:13.365133 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 573/1156] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:56:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:56:13.370181 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dc7b22-7f6b-437f-b099-fe425d1c0deb None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:13.467191 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dc7b22-7f6b-437f-b099-fe425d1c0deb None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:13.468622 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:13.502926 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39489dae-6559-46da-a4e3-3ddc9fd3f083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:13.503528 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39489dae-6559-46da-a4e3-3ddc9fd3f083 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:13.503806 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39489dae-6559-46da-a4e3-3ddc9fd3f083 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:13.504081 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39489dae-6559-46da-a4e3-3ddc9fd3f083 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:13.504736 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39489dae-6559-46da-a4e3-3ddc9fd3f083 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:56:13.505037 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39489dae-6559-46da-a4e3-3ddc9fd3f083 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:13.506405 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39489dae-6559-46da-a4e3-3ddc9fd3f083 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:13.514201 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 584/1157] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:56:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:56:13.520324 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68e980-099c-4cf9-a912-cd0bd2e89373 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:13.620280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68e980-099c-4cf9-a912-cd0bd2e89373 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:13.644921 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:13.653768 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbdcf9b-bcd8-47f8-9fdb-2819f2b29a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:13.654395 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdcf9b-bcd8-47f8-9fdb-2819f2b29a09 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:13.654678 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdcf9b-bcd8-47f8-9fdb-2819f2b29a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:13.654991 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdcf9b-bcd8-47f8-9fdb-2819f2b29a09 None admin] PATH_INFO: `/v3/users/e5441b73c4f94560873731648e44d1b9` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:13.656108 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbdcf9b-bcd8-47f8-9fdb-2819f2b29a09 None admin] RBAC: Authorizing `identity:get_user(user_id=e5441b73c4f94560873731648e44d1b9)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:13.657547 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbdcf9b-bcd8-47f8-9fdb-2819f2b29a09 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:13.659158 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 574/1158] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:56:13 2022] GET /identity/v3/users/e5441b73c4f94560873731648e44d1b9 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:56:13.664932 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39489dae-6559-46da-a4e3-3ddc9fd3f083 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:13.762236 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39489dae-6559-46da-a4e3-3ddc9fd3f083 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:13.794598 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc43bab-9f62-4d64-beac-ee26bf4e8494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:13.795233 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc43bab-9f62-4d64-beac-ee26bf4e8494 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:13.795515 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc43bab-9f62-4d64-beac-ee26bf4e8494 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:13.795783 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc43bab-9f62-4d64-beac-ee26bf4e8494 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:13.826390 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:13.863259 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc43bab-9f62-4d64-beac-ee26bf4e8494 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:13.864713 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc43bab-9f62-4d64-beac-ee26bf4e8494 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:13.967004 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 585/1159] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:56:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 14:56:13.972999 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbdcf9b-bcd8-47f8-9fdb-2819f2b29a09 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:14.070719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbdcf9b-bcd8-47f8-9fdb-2819f2b29a09 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:14.103302 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7cf88d-102a-4c70-a794-0bff5398527b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:14.103923 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cf88d-102a-4c70-a794-0bff5398527b None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:14.104204 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cf88d-102a-4c70-a794-0bff5398527b None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:14.104478 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cf88d-102a-4c70-a794-0bff5398527b None admin] PATH_INFO: `/v3/projects/75cdab48e60f473f9e4fc6e63b3d3a45` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:14.105592 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7cf88d-102a-4c70-a794-0bff5398527b None admin] RBAC: Authorizing `identity:get_project(project_id=75cdab48e60f473f9e4fc6e63b3d3a45)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:14.107230 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7cf88d-102a-4c70-a794-0bff5398527b None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:14.108856 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 575/1160] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:56:13 2022] GET /identity/v3/projects/75cdab48e60f473f9e4fc6e63b3d3a45 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:56:14.115036 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc43bab-9f62-4d64-beac-ee26bf4e8494 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:14.177171 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:14.215237 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc43bab-9f62-4d64-beac-ee26bf4e8494 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:14.248625 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95bd1e1-68fd-4b11-a4e4-658ff5fd1ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:14.249221 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95bd1e1-68fd-4b11-a4e4-658ff5fd1ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:14.249501 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95bd1e1-68fd-4b11-a4e4-658ff5fd1ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:14.249774 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95bd1e1-68fd-4b11-a4e4-658ff5fd1ed4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:14.250494 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95bd1e1-68fd-4b11-a4e4-658ff5fd1ed4 None admin] RBAC: Adding query filter params (role.id=99e8fe7df8e7446680a81d4eaaa1cb68, scope.project.id=75cdab48e60f473f9e4fc6e63b3d3a45, user.id=e5441b73c4f94560873731648e44d1b9) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:56:14.250834 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95bd1e1-68fd-4b11-a4e4-658ff5fd1ed4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:14.252418 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95bd1e1-68fd-4b11-a4e4-658ff5fd1ed4 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:14.258100 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 586/1161] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:56:14 2022] GET /identity/v3/role_assignments?user.id=e5441b73c4f94560873731648e44d1b9&scope.project.id=75cdab48e60f473f9e4fc6e63b3d3a45&role.id=99e8fe7df8e7446680a81d4eaaa1cb68 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:56:16.414180 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d6d3cb-48c9-4779-ab46-ac38cb26d386 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:16.414707 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d6d3cb-48c9-4779-ab46-ac38cb26d386 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:16.414707 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d6d3cb-48c9-4779-ab46-ac38cb26d386 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:16.416009 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 576/1162] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:56:16.421738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37137b26-d2e4-4941-a45a-2af8594bd31c None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:16.422142 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37137b26-d2e4-4941-a45a-2af8594bd31c None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:16.422284 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37137b26-d2e4-4941-a45a-2af8594bd31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:16.440340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-37137b26-d2e4-4941-a45a-2af8594bd31c None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:56:16.441525 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-37137b26-d2e4-4941-a45a-2af8594bd31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:56:16.506024 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 587/1163] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:56:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:56:16.779103 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d6d3cb-48c9-4779-ab46-ac38cb26d386 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:16.841400 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:16.881613 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d6d3cb-48c9-4779-ab46-ac38cb26d386 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:16.913659 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a483c02-42d7-4ddc-8618-96f4b08f9329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:16.914247 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a483c02-42d7-4ddc-8618-96f4b08f9329 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:16.914529 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a483c02-42d7-4ddc-8618-96f4b08f9329 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:16.914836 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a483c02-42d7-4ddc-8618-96f4b08f9329 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:16.983092 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a483c02-42d7-4ddc-8618-96f4b08f9329 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:16.984541 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a483c02-42d7-4ddc-8618-96f4b08f9329 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:17.042067 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:17.081153 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 577/1164] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:56:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 14:56:17.087203 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37137b26-d2e4-4941-a45a-2af8594bd31c None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:17.144449 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:17.186541 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37137b26-d2e4-4941-a45a-2af8594bd31c None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:17.220184 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae756af-b50a-442c-aaad-3bd6b87f023f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:17.220815 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae756af-b50a-442c-aaad-3bd6b87f023f None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:17.221096 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae756af-b50a-442c-aaad-3bd6b87f023f None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:17.221369 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae756af-b50a-442c-aaad-3bd6b87f023f None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:17.228164 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae756af-b50a-442c-aaad-3bd6b87f023f None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:17.229617 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae756af-b50a-442c-aaad-3bd6b87f023f None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:17.242313 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-9ae756af-b50a-442c-aaad-3bd6b87f023f None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 11 14:56:17.242313 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:56:17.242313 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 11 14:56:17.242313 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 11 14:56:17.242313 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:56:17.242313 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:56:17.242313 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:56:17.242313 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:56:17.242313 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 11 14:56:17.242313 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:56:17.242313 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:56:17.242313 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:56:17.242313 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:56:17.242313 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:56:17.242313 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:56:17.242313 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:56:17.242313 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:56:17.243607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:56:17.243607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:56:17.243607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:56:17.243607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:56:17.243607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:56:17.243607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:56:17.243607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:56:17.243607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:56:17.243607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:56:17.243607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:56:17.243607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:56:17.243607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:56:17.243607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:56:17.243607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:56:17.243607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:56:17.243607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:56:17.244680 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:56:17.244680 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:56:17.244680 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:56:17.244680 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:56:17.244680 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:56:17.244680 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:56:17.244680 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:56:17.244680 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:56:17.244680 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:56:17.244680 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:56:17.244680 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 11 14:56:17.244680 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:56:17.244680 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 11 14:56:17.244680 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:56:17.244680 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 588/1165] 10.222.0.33 () {56 vars in 1117 bytes} [Sun Sep 11 14:56:17 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:56:17.249660 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a483c02-42d7-4ddc-8618-96f4b08f9329 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:17.348575 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a483c02-42d7-4ddc-8618-96f4b08f9329 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:17.381461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4fe0ac-5cd3-4dd1-89ff-e170eed45e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:17.382058 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4fe0ac-5cd3-4dd1-89ff-e170eed45e0c None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:17.382340 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4fe0ac-5cd3-4dd1-89ff-e170eed45e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:17.382617 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4fe0ac-5cd3-4dd1-89ff-e170eed45e0c None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:17.383282 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4fe0ac-5cd3-4dd1-89ff-e170eed45e0c None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:56:17.383588 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4fe0ac-5cd3-4dd1-89ff-e170eed45e0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:17.384957 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4fe0ac-5cd3-4dd1-89ff-e170eed45e0c None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:17.392724 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 578/1166] 10.222.0.33 () {56 vars in 1115 bytes} [Sun Sep 11 14:56:17 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:56:19.524110 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7da27d-e5a5-49e2-8c96-a4b772c320bf None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:19.524648 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7da27d-e5a5-49e2-8c96-a4b772c320bf None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:19.524648 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7da27d-e5a5-49e2-8c96-a4b772c320bf None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:19.525990 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 589/1167] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:56:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:56:19.532106 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c4ea7-337c-4f48-b012-a12bef996267 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:19.532550 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c4ea7-337c-4f48-b012-a12bef996267 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:19.532652 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c4ea7-337c-4f48-b012-a12bef996267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:19.551441 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-7a7c4ea7-337c-4f48-b012-a12bef996267 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:56:19.552613 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-7a7c4ea7-337c-4f48-b012-a12bef996267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:56:19.572238 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:19.618102 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 579/1168] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:56:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:56:19.890860 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7da27d-e5a5-49e2-8c96-a4b772c320bf None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:19.990825 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7da27d-e5a5-49e2-8c96-a4b772c320bf None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:20.007690 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:20.024081 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c9507-6be3-4db7-afed-2acec22fa023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:20.024747 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c9507-6be3-4db7-afed-2acec22fa023 None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:20.025029 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c9507-6be3-4db7-afed-2acec22fa023 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:20.025302 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c9507-6be3-4db7-afed-2acec22fa023 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:20.025971 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c9507-6be3-4db7-afed-2acec22fa023 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:20.027990 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c9507-6be3-4db7-afed-2acec22fa023 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:20.043897 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 590/1169] 10.222.0.33 () {60 vars in 1138 bytes} [Sun Sep 11 14:56:19 2022] POST /identity/v3/domains => generated 267 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:56:22.207383 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250bde7-ef2f-4062-ba23-e00f91059671 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:22.207938 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250bde7-ef2f-4062-ba23-e00f91059671 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:22.207938 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250bde7-ef2f-4062-ba23-e00f91059671 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:22.209268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 580/1170] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:56:22.214526 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2ed7b-a7df-4feb-b5f4-d13852e2c4a7 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:22.214995 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2ed7b-a7df-4feb-b5f4-d13852e2c4a7 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:22.215171 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2ed7b-a7df-4feb-b5f4-d13852e2c4a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:22.233070 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-12f2ed7b-a7df-4feb-b5f4-d13852e2c4a7 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:56:22.234317 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-12f2ed7b-a7df-4feb-b5f4-d13852e2c4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:56:22.299758 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 591/1171] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:56:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:56:22.570289 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1250bde7-ef2f-4062-ba23-e00f91059671 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:22.668722 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1250bde7-ef2f-4062-ba23-e00f91059671 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:22.702015 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bec49c-a9d9-4ddc-87f7-2fc138cdfbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:22.702628 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bec49c-a9d9-4ddc-87f7-2fc138cdfbed None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:22.702937 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bec49c-a9d9-4ddc-87f7-2fc138cdfbed None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:22.703211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bec49c-a9d9-4ddc-87f7-2fc138cdfbed None admin] PATH_INFO: `/v3/domains/ce8e1e296c994241b6ded15c6c708ee7` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:22.713175 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bec49c-a9d9-4ddc-87f7-2fc138cdfbed None admin] RBAC: Authorizing `identity:get_domain(domain_id=ce8e1e296c994241b6ded15c6c708ee7)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:22.714611 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bec49c-a9d9-4ddc-87f7-2fc138cdfbed None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:22.716350 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 581/1172] 10.222.0.33 () {56 vars in 1183 bytes} [Sun Sep 11 14:56:22 2022] GET /identity/v3/domains/ce8e1e296c994241b6ded15c6c708ee7 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:56:22.722259 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2ed7b-a7df-4feb-b5f4-d13852e2c4a7 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:22.818249 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2ed7b-a7df-4feb-b5f4-d13852e2c4a7 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:22.827189 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:22.851353 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc4407-9452-4490-b82d-f81fa70d2c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:22.851946 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc4407-9452-4490-b82d-f81fa70d2c02 None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:22.852223 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc4407-9452-4490-b82d-f81fa70d2c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:22.852492 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc4407-9452-4490-b82d-f81fa70d2c02 None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:22.853136 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bc4407-9452-4490-b82d-f81fa70d2c02 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:22.854566 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bc4407-9452-4490-b82d-f81fa70d2c02 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:22.880507 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 592/1173] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 11 14:56:22 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 14:56:25.038336 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71189714-eb3a-4a24-9dde-ea46041485ba None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:25.038845 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71189714-eb3a-4a24-9dde-ea46041485ba None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:25.038967 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71189714-eb3a-4a24-9dde-ea46041485ba None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:25.040242 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 582/1174] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:56:25.046063 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386212d9-1de3-47b0-a2eb-ed2991827df1 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:25.046444 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386212d9-1de3-47b0-a2eb-ed2991827df1 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:25.046608 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386212d9-1de3-47b0-a2eb-ed2991827df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:25.065471 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-386212d9-1de3-47b0-a2eb-ed2991827df1 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:56:25.066662 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-386212d9-1de3-47b0-a2eb-ed2991827df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:56:25.132006 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 593/1175] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:56:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:56:25.406897 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71189714-eb3a-4a24-9dde-ea46041485ba None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:25.425893 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:25.508740 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71189714-eb3a-4a24-9dde-ea46041485ba None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:25.541346 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c110aa-99a2-495b-ae27-098cde49ba87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:25.541957 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c110aa-99a2-495b-ae27-098cde49ba87 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:25.542234 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c110aa-99a2-495b-ae27-098cde49ba87 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:25.542498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c110aa-99a2-495b-ae27-098cde49ba87 None admin] PATH_INFO: `/v3/domains/ce8e1e296c994241b6ded15c6c708ee7` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:25.544492 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c110aa-99a2-495b-ae27-098cde49ba87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ce8e1e296c994241b6ded15c6c708ee7)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:25.545925 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c110aa-99a2-495b-ae27-098cde49ba87 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:25.547514 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 583/1176] 10.222.0.33 () {56 vars in 1183 bytes} [Sun Sep 11 14:56:25 2022] GET /identity/v3/domains/ce8e1e296c994241b6ded15c6c708ee7 => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:56:25.553285 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386212d9-1de3-47b0-a2eb-ed2991827df1 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:25.642504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:25.653384 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386212d9-1de3-47b0-a2eb-ed2991827df1 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:25.686574 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f5956c-ebbf-4ddb-b3e5-71e62257cc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:25.687153 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f5956c-ebbf-4ddb-b3e5-71e62257cc21 None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:25.687434 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f5956c-ebbf-4ddb-b3e5-71e62257cc21 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:25.687706 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f5956c-ebbf-4ddb-b3e5-71e62257cc21 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:25.688347 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f5956c-ebbf-4ddb-b3e5-71e62257cc21 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:25.689794 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f5956c-ebbf-4ddb-b3e5-71e62257cc21 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:25.710907 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 594/1177] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 14:56:25 2022] POST /identity/v3/users => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:56:27.894075 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0a356-fda1-4162-8465-4a60c4673bcb None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:27.894650 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0a356-fda1-4162-8465-4a60c4673bcb None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:27.894650 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0a356-fda1-4162-8465-4a60c4673bcb None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:27.896023 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 584/1178] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:56:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:56:27.901172 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bccc0-ed34-4dd6-b17b-50afb6c7b9e5 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:27.901581 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bccc0-ed34-4dd6-b17b-50afb6c7b9e5 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:27.901697 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bccc0-ed34-4dd6-b17b-50afb6c7b9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:27.920116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-7a0bccc0-ed34-4dd6-b17b-50afb6c7b9e5 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:56:27.921290 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7a0bccc0-ed34-4dd6-b17b-50afb6c7b9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:56:27.987102 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 595/1179] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:56:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:56:28.255834 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0a356-fda1-4162-8465-4a60c4673bcb None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:28.339564 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:28.357750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0a356-fda1-4162-8465-4a60c4673bcb None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:28.390059 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f1aca8-f264-43ce-80ec-5669b5d590c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:28.390675 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1aca8-f264-43ce-80ec-5669b5d590c1 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:28.390984 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1aca8-f264-43ce-80ec-5669b5d590c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:28.391260 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1aca8-f264-43ce-80ec-5669b5d590c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:28.401678 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f1aca8-f264-43ce-80ec-5669b5d590c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:56:28.407301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-65f1aca8-f264-43ce-80ec-5669b5d590c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:56:28.407301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:56:28.407301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:56:28.407301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:56:28.407301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:56:28.407301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:56:28.407301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:56:28.407301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:56:28.407301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:56:28.407301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:56:28.407301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:56:28.407301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:56:28.407301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:56:28.407301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:56:28.407301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:56:28.407301 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:56:28.408412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:56:28.408412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:56:28.408412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:56:28.408412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:56:28.408412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:56:28.408412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:56:28.408412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:56:28.408412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:56:28.408412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:56:28.408412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:56:28.408412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:56:28.408412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:56:28.408412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:56:28.408412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:56:28.408412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:56:28.408412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:56:28.409441 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:56:28.409441 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:56:28.409441 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:56:28.409441 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:56:28.409441 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:56:28.409441 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:56:28.409441 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:56:28.409441 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:56:28.409441 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:56:28.409441 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:56:28.409441 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:56:28.409441 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:56:28.409441 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 585/1180] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:56:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:56:28.414368 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0bccc0-ed34-4dd6-b17b-50afb6c7b9e5 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:28.488341 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:28.511213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0bccc0-ed34-4dd6-b17b-50afb6c7b9e5 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:28.544599 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338374ea-0907-4504-bddd-27855cc68fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:28.545197 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338374ea-0907-4504-bddd-27855cc68fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:28.545471 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338374ea-0907-4504-bddd-27855cc68fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:28.545738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338374ea-0907-4504-bddd-27855cc68fb8 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:28.546387 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338374ea-0907-4504-bddd-27855cc68fb8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:56:28.546680 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338374ea-0907-4504-bddd-27855cc68fb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:28.548081 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338374ea-0907-4504-bddd-27855cc68fb8 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:28.555778 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 596/1181] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:56:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:56:28.562047 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f1aca8-f264-43ce-80ec-5669b5d590c1 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:28.660842 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f1aca8-f264-43ce-80ec-5669b5d590c1 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:28.693064 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd6fed-a696-4a80-90bb-8264cd4d2efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:28.693679 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd6fed-a696-4a80-90bb-8264cd4d2efd None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:28.693960 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd6fed-a696-4a80-90bb-8264cd4d2efd None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:28.694233 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd6fed-a696-4a80-90bb-8264cd4d2efd None admin] PATH_INFO: `/v3/users/8e471d841eac44b3baaad50a5ebe1792` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:28.707342 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbd6fed-a696-4a80-90bb-8264cd4d2efd None admin] RBAC: Authorizing `identity:get_user(user_id=8e471d841eac44b3baaad50a5ebe1792)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:28.708783 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbd6fed-a696-4a80-90bb-8264cd4d2efd None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:28.710352 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 586/1182] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:56:28 2022] GET /identity/v3/users/8e471d841eac44b3baaad50a5ebe1792 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:56:28.716807 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338374ea-0907-4504-bddd-27855cc68fb8 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:28.813430 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338374ea-0907-4504-bddd-27855cc68fb8 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:28.846005 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d72c8a-73ec-4a5b-921b-f2b591f7864b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:28.846599 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d72c8a-73ec-4a5b-921b-f2b591f7864b None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:28.846907 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d72c8a-73ec-4a5b-921b-f2b591f7864b None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:28.847179 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d72c8a-73ec-4a5b-921b-f2b591f7864b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:28.855132 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:28.913634 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d72c8a-73ec-4a5b-921b-f2b591f7864b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:28.915115 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d72c8a-73ec-4a5b-921b-f2b591f7864b None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:29.011126 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 597/1183] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:56:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 14:56:29.017068 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd6fed-a696-4a80-90bb-8264cd4d2efd None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:29.020041 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:29.118054 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd6fed-a696-4a80-90bb-8264cd4d2efd None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:29.150353 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e06b833-9a70-42d7-8936-2605fcae3a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:29.151003 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06b833-9a70-42d7-8936-2605fcae3a4e None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:29.151284 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06b833-9a70-42d7-8936-2605fcae3a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:29.151555 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06b833-9a70-42d7-8936-2605fcae3a4e None admin] PATH_INFO: `/v3/projects/9c4c23e226e54eeaa845b8afe4b0a570` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:29.153234 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e06b833-9a70-42d7-8936-2605fcae3a4e None admin] RBAC: Authorizing `identity:get_project(project_id=9c4c23e226e54eeaa845b8afe4b0a570)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:29.154655 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e06b833-9a70-42d7-8936-2605fcae3a4e None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:29.156283 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 587/1184] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:56:29 2022] GET /identity/v3/projects/9c4c23e226e54eeaa845b8afe4b0a570 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:56:29.162444 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d72c8a-73ec-4a5b-921b-f2b591f7864b None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:29.201034 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:29.259861 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d72c8a-73ec-4a5b-921b-f2b591f7864b None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:29.293756 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5342e5d6-495e-413b-885f-280db263c930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:29.294346 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342e5d6-495e-413b-885f-280db263c930 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:29.294627 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342e5d6-495e-413b-885f-280db263c930 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:29.294925 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342e5d6-495e-413b-885f-280db263c930 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:29.295640 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5342e5d6-495e-413b-885f-280db263c930 None admin] RBAC: Adding query filter params (role.id=99e8fe7df8e7446680a81d4eaaa1cb68, scope.project.id=9c4c23e226e54eeaa845b8afe4b0a570, user.id=8e471d841eac44b3baaad50a5ebe1792) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:56:29.295947 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5342e5d6-495e-413b-885f-280db263c930 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:29.297381 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5342e5d6-495e-413b-885f-280db263c930 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:29.302589 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 598/1185] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:56:29 2022] GET /identity/v3/role_assignments?user.id=8e471d841eac44b3baaad50a5ebe1792&scope.project.id=9c4c23e226e54eeaa845b8afe4b0a570&role.id=99e8fe7df8e7446680a81d4eaaa1cb68 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:56:31.418448 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581738d2-f1ce-4d6d-bd82-b4b7096c4992 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:31.419008 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581738d2-f1ce-4d6d-bd82-b4b7096c4992 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:31.419008 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581738d2-f1ce-4d6d-bd82-b4b7096c4992 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:31.420772 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 588/1186] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:56:31.425892 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a04338-001d-4f5d-894a-9886178f35c0 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:31.426311 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a04338-001d-4f5d-894a-9886178f35c0 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:31.426438 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a04338-001d-4f5d-894a-9886178f35c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:31.445218 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-09a04338-001d-4f5d-894a-9886178f35c0 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:56:31.446412 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-09a04338-001d-4f5d-894a-9886178f35c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:56:31.512514 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 599/1187] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:56:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:56:31.784405 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581738d2-f1ce-4d6d-bd82-b4b7096c4992 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:31.853850 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:31.885835 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581738d2-f1ce-4d6d-bd82-b4b7096c4992 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:31.918936 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58a9e1-21c6-4cc7-b7e7-801b85a2f227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:31.919556 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58a9e1-21c6-4cc7-b7e7-801b85a2f227 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:31.919837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58a9e1-21c6-4cc7-b7e7-801b85a2f227 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:31.920102 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58a9e1-21c6-4cc7-b7e7-801b85a2f227 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:31.929723 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58a9e1-21c6-4cc7-b7e7-801b85a2f227 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:56:31.935039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-0f58a9e1-21c6-4cc7-b7e7-801b85a2f227 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:56:31.935039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:56:31.935039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:56:31.935039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:56:31.935039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:56:31.935039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:56:31.935039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:56:31.935039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:56:31.935039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:56:31.935039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:56:31.935039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:56:31.935039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:56:31.935039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:56:31.935039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:56:31.935039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:56:31.935039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:56:31.936138 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:56:31.936138 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:56:31.936138 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:56:31.936138 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:56:31.936138 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:56:31.936138 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:56:31.936138 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:56:31.936138 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:56:31.936138 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:56:31.936138 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:56:31.936138 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:56:31.936138 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:56:31.936138 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:56:31.936138 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:56:31.936138 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:56:31.936138 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:56:31.937253 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:56:31.937253 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:56:31.937253 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:56:31.937253 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:56:31.937253 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:56:31.937253 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:56:31.937253 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:56:31.937253 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:56:31.937253 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:56:31.937253 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:56:31.937253 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:56:31.937253 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:56:31.937253 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 589/1188] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:56:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:56:31.942061 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a04338-001d-4f5d-894a-9886178f35c0 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:31.972293 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:32.040878 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a04338-001d-4f5d-894a-9886178f35c0 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:32.074576 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea7a70-97b8-4020-bb74-2c0c2d9256ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:32.075205 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea7a70-97b8-4020-bb74-2c0c2d9256ba None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:32.075485 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea7a70-97b8-4020-bb74-2c0c2d9256ba None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:32.075782 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea7a70-97b8-4020-bb74-2c0c2d9256ba None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:32.076442 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea7a70-97b8-4020-bb74-2c0c2d9256ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:56:32.076739 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea7a70-97b8-4020-bb74-2c0c2d9256ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:32.078121 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea7a70-97b8-4020-bb74-2c0c2d9256ba None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:32.085965 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 600/1189] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:56:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:56:32.092423 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58a9e1-21c6-4cc7-b7e7-801b85a2f227 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:32.189964 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58a9e1-21c6-4cc7-b7e7-801b85a2f227 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:32.218031 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:32.224252 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2a962-590a-4571-97b3-db6be55ac6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:32.224879 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2a962-590a-4571-97b3-db6be55ac6ec None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:32.225157 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2a962-590a-4571-97b3-db6be55ac6ec None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:32.225430 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2a962-590a-4571-97b3-db6be55ac6ec None admin] PATH_INFO: `/v3/users/8e471d841eac44b3baaad50a5ebe1792` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:32.226539 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2a962-590a-4571-97b3-db6be55ac6ec None admin] RBAC: Authorizing `identity:get_user(user_id=8e471d841eac44b3baaad50a5ebe1792)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:32.228009 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2a962-590a-4571-97b3-db6be55ac6ec None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:32.229571 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 590/1190] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:56:32 2022] GET /identity/v3/users/8e471d841eac44b3baaad50a5ebe1792 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:56:32.235827 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea7a70-97b8-4020-bb74-2c0c2d9256ba None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:32.329832 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:32.336793 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea7a70-97b8-4020-bb74-2c0c2d9256ba None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:32.369517 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b73a6-66f2-461c-8fbc-490b637f08ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:32.370135 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b73a6-66f2-461c-8fbc-490b637f08ac None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:32.370411 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b73a6-66f2-461c-8fbc-490b637f08ac None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:32.370683 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b73a6-66f2-461c-8fbc-490b637f08ac None admin] PATH_INFO: `/v3/projects/9c4c23e226e54eeaa845b8afe4b0a570` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:32.371907 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b73a6-66f2-461c-8fbc-490b637f08ac None admin] RBAC: Authorizing `identity:get_project(project_id=9c4c23e226e54eeaa845b8afe4b0a570)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:32.373337 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b73a6-66f2-461c-8fbc-490b637f08ac None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:32.374964 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 601/1191] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:56:32 2022] GET /identity/v3/projects/9c4c23e226e54eeaa845b8afe4b0a570 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:56:32.380916 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2a962-590a-4571-97b3-db6be55ac6ec None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:32.478790 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2a962-590a-4571-97b3-db6be55ac6ec None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:32.512479 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a5c1ee-a128-4c70-b299-d9f580bc805d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:32.513125 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a5c1ee-a128-4c70-b299-d9f580bc805d None admin] REQUEST_METHOD: `PUT` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:32.513407 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a5c1ee-a128-4c70-b299-d9f580bc805d None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:32.513681 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a5c1ee-a128-4c70-b299-d9f580bc805d None admin] PATH_INFO: `/v3/projects/9c4c23e226e54eeaa845b8afe4b0a570/users/8e471d841eac44b3baaad50a5ebe1792/roles/99e8fe7df8e7446680a81d4eaaa1cb68` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:32.515600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a5c1ee-a128-4c70-b299-d9f580bc805d None admin] RBAC: Authorizing `identity:create_grant(project_id=9c4c23e226e54eeaa845b8afe4b0a570, user_id=8e471d841eac44b3baaad50a5ebe1792, role_id=99e8fe7df8e7446680a81d4eaaa1cb68)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:32.517033 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a5c1ee-a128-4c70-b299-d9f580bc805d None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:32.530304 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 591/1192] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 14:56:32 2022] PUT /identity/v3/projects/9c4c23e226e54eeaa845b8afe4b0a570/users/8e471d841eac44b3baaad50a5ebe1792/roles/99e8fe7df8e7446680a81d4eaaa1cb68 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:56:34.646605 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa3e6a-f99a-49aa-a828-19619bfb1cd7 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:34.647212 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa3e6a-f99a-49aa-a828-19619bfb1cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:34.647212 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa3e6a-f99a-49aa-a828-19619bfb1cd7 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:34.648521 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 602/1193] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:56:34.653821 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2c53a-6a7d-465a-91cd-69c38e204b2f None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:34.654200 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2c53a-6a7d-465a-91cd-69c38e204b2f None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:34.654362 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2c53a-6a7d-465a-91cd-69c38e204b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:34.672581 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-dda2c53a-6a7d-465a-91cd-69c38e204b2f None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:56:34.673756 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-dda2c53a-6a7d-465a-91cd-69c38e204b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:56:34.726129 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:34.740794 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 592/1194] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:56:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:56:35.010562 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fa3e6a-f99a-49aa-a828-19619bfb1cd7 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:35.111040 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fa3e6a-f99a-49aa-a828-19619bfb1cd7 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:35.144251 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84b5f4f-2218-420d-8964-be0e6f4a3837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:35.144867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b5f4f-2218-420d-8964-be0e6f4a3837 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:35.145149 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b5f4f-2218-420d-8964-be0e6f4a3837 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:35.145423 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b5f4f-2218-420d-8964-be0e6f4a3837 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:35.156421 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84b5f4f-2218-420d-8964-be0e6f4a3837 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:56:35.161216 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-c84b5f4f-2218-420d-8964-be0e6f4a3837 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:56:35.161216 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:56:35.161216 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:56:35.161216 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:56:35.161216 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:56:35.161216 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:56:35.161216 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:56:35.161216 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:56:35.161216 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:56:35.161216 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:56:35.161216 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:56:35.161216 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:56:35.161216 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:56:35.161216 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:56:35.161216 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:56:35.161216 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:56:35.162393 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:56:35.162393 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:56:35.162393 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:56:35.162393 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:56:35.162393 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:56:35.162393 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:56:35.162393 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:56:35.162393 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:56:35.162393 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:56:35.162393 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:56:35.162393 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:56:35.162393 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:56:35.162393 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:56:35.162393 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:56:35.162393 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:56:35.162393 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:56:35.163340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:56:35.163340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:56:35.163340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:56:35.163340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:56:35.163340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:56:35.163340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:56:35.163340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:56:35.163340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:56:35.163340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:56:35.163340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:56:35.163340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:56:35.163340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:56:35.163340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 603/1195] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:56:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:56:35.168902 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda2c53a-6a7d-465a-91cd-69c38e204b2f None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:35.264363 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda2c53a-6a7d-465a-91cd-69c38e204b2f None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:35.297365 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9a2a9d-5695-43ba-a23a-01d94f6ee441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:35.297960 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9a2a9d-5695-43ba-a23a-01d94f6ee441 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:35.298240 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9a2a9d-5695-43ba-a23a-01d94f6ee441 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:35.298511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9a2a9d-5695-43ba-a23a-01d94f6ee441 None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:35.299195 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9a2a9d-5695-43ba-a23a-01d94f6ee441 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:56:35.299497 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9a2a9d-5695-43ba-a23a-01d94f6ee441 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:35.300860 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9a2a9d-5695-43ba-a23a-01d94f6ee441 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:35.309024 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 593/1196] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:56:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:56:35.315156 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84b5f4f-2218-420d-8964-be0e6f4a3837 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:35.340306 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:35.414369 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84b5f4f-2218-420d-8964-be0e6f4a3837 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:35.447918 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a6dcae-e6fc-46b7-a65e-4a97d141d949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:35.448540 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6dcae-e6fc-46b7-a65e-4a97d141d949 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:35.448822 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6dcae-e6fc-46b7-a65e-4a97d141d949 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:35.449179 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6dcae-e6fc-46b7-a65e-4a97d141d949 None admin] PATH_INFO: `/v3/users/8e471d841eac44b3baaad50a5ebe1792` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:35.450337 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a6dcae-e6fc-46b7-a65e-4a97d141d949 None admin] RBAC: Authorizing `identity:get_user(user_id=8e471d841eac44b3baaad50a5ebe1792)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:35.451820 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a6dcae-e6fc-46b7-a65e-4a97d141d949 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:35.453387 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 604/1197] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 14:56:35 2022] GET /identity/v3/users/8e471d841eac44b3baaad50a5ebe1792 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:56:35.459096 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9a2a9d-5695-43ba-a23a-01d94f6ee441 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:35.508588 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:35.558208 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9a2a9d-5695-43ba-a23a-01d94f6ee441 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:35.591318 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f4c7c-8830-4594-9efe-fe7909418f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:35.591920 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f4c7c-8830-4594-9efe-fe7909418f81 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:35.592201 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f4c7c-8830-4594-9efe-fe7909418f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:35.592473 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f4c7c-8830-4594-9efe-fe7909418f81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:35.657633 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31f4c7c-8830-4594-9efe-fe7909418f81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:35.659116 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31f4c7c-8830-4594-9efe-fe7909418f81 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:35.708798 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:35.758682 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 594/1198] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:56:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 14:56:35.764926 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a6dcae-e6fc-46b7-a65e-4a97d141d949 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:35.854014 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:35.864471 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a6dcae-e6fc-46b7-a65e-4a97d141d949 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:35.897394 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7406b3-92d4-4c96-96dd-879069054b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:35.898007 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7406b3-92d4-4c96-96dd-879069054b5b None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:35.898286 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7406b3-92d4-4c96-96dd-879069054b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:35.898556 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7406b3-92d4-4c96-96dd-879069054b5b None admin] PATH_INFO: `/v3/projects/9c4c23e226e54eeaa845b8afe4b0a570` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:35.899684 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7406b3-92d4-4c96-96dd-879069054b5b None admin] RBAC: Authorizing `identity:get_project(project_id=9c4c23e226e54eeaa845b8afe4b0a570)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:35.901119 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7406b3-92d4-4c96-96dd-879069054b5b None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:35.902769 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 605/1199] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 14:56:35 2022] GET /identity/v3/projects/9c4c23e226e54eeaa845b8afe4b0a570 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:56:35.908862 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f4c7c-8830-4594-9efe-fe7909418f81 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:56:36.006619 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f4c7c-8830-4594-9efe-fe7909418f81 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:56:36.039665 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65522bf7-8a19-47e0-93c5-82c99c62bb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:56:36.040261 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65522bf7-8a19-47e0-93c5-82c99c62bb39 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:56:36.040540 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65522bf7-8a19-47e0-93c5-82c99c62bb39 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:56:36.040810 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65522bf7-8a19-47e0-93c5-82c99c62bb39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:56:36.041517 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65522bf7-8a19-47e0-93c5-82c99c62bb39 None admin] RBAC: Adding query filter params (role.id=99e8fe7df8e7446680a81d4eaaa1cb68, scope.project.id=9c4c23e226e54eeaa845b8afe4b0a570, user.id=8e471d841eac44b3baaad50a5ebe1792) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:56:36.041821 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65522bf7-8a19-47e0-93c5-82c99c62bb39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:56:36.043769 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65522bf7-8a19-47e0-93c5-82c99c62bb39 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:56:36.049321 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 595/1200] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:56:35 2022] GET /identity/v3/role_assignments?user.id=8e471d841eac44b3baaad50a5ebe1792&scope.project.id=9c4c23e226e54eeaa845b8afe4b0a570&role.id=99e8fe7df8e7446680a81d4eaaa1cb68 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:56:45.893360 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=21 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:56:46.046782 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=93 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:31.863113 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618aad2-4a33-4615-b4cc-9f08acdf041f None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:31.863745 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618aad2-4a33-4615-b4cc-9f08acdf041f None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:31.863745 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618aad2-4a33-4615-b4cc-9f08acdf041f None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:31.865524 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 606/1201] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:57:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:57:31.872462 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df13b4c-8e61-45d5-b48e-7292b4af398a None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:31.872973 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df13b4c-8e61-45d5-b48e-7292b4af398a None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:31.873093 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df13b4c-8e61-45d5-b48e-7292b4af398a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:31.903000 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-5df13b4c-8e61-45d5-b48e-7292b4af398a None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:57:31.904848 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-5df13b4c-8e61-45d5-b48e-7292b4af398a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:57:32.010914 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 596/1202] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:57:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:57:32.284087 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618aad2-4a33-4615-b4cc-9f08acdf041f None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:32.428005 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618aad2-4a33-4615-b4cc-9f08acdf041f None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:32.465319 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d65171-6562-47c9-8316-3fdd0797c89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:32.466903 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d65171-6562-47c9-8316-3fdd0797c89e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:32.466903 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d65171-6562-47c9-8316-3fdd0797c89e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:32.466903 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d65171-6562-47c9-8316-3fdd0797c89e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:32.481876 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d65171-6562-47c9-8316-3fdd0797c89e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:32.483444 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d65171-6562-47c9-8316-3fdd0797c89e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:32.499461 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-b3d65171-6562-47c9-8316-3fdd0797c89e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:57:32.499461 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:32.499461 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:32.499461 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:32.499461 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:32.499461 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:32.499461 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:32.499461 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:32.499461 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:32.499461 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:32.499461 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:32.499461 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:32.499461 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:57:32.499461 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:57:32.499461 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:57:32.499461 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:57:32.500593 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:32.500593 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:32.500593 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:32.500593 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:32.500593 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:32.500593 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:32.500593 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:32.500593 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:32.500593 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:32.500593 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:32.500593 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:32.500593 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:32.500593 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:32.500593 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:32.500593 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:32.500593 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:32.500593 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:57:32.501588 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:57:32.501588 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:57:32.501588 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:57:32.507107 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 607/1203] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:57:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:57:32.508271 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df13b4c-8e61-45d5-b48e-7292b4af398a None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:32.636703 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df13b4c-8e61-45d5-b48e-7292b4af398a None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:32.680388 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:32.692242 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e00f798-4eee-4464-a67f-271efba50346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:32.692958 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00f798-4eee-4464-a67f-271efba50346 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:32.693235 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00f798-4eee-4464-a67f-271efba50346 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:32.693554 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00f798-4eee-4464-a67f-271efba50346 None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:32.694275 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e00f798-4eee-4464-a67f-271efba50346 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:32.694589 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e00f798-4eee-4464-a67f-271efba50346 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:32.696876 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e00f798-4eee-4464-a67f-271efba50346 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:32.715388 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 597/1204] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:57:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:32.720765 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d65171-6562-47c9-8316-3fdd0797c89e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:32.790492 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:32.857189 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d65171-6562-47c9-8316-3fdd0797c89e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:32.898632 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd745213-7b30-4f04-9dd9-9e531560c37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:32.899285 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd745213-7b30-4f04-9dd9-9e531560c37d None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:32.899461 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd745213-7b30-4f04-9dd9-9e531560c37d None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:32.899740 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd745213-7b30-4f04-9dd9-9e531560c37d None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:32.900509 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd745213-7b30-4f04-9dd9-9e531560c37d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:32.902761 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd745213-7b30-4f04-9dd9-9e531560c37d None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:32.930262 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 608/1205] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 14:57:32 2022] POST /identity/v3/users => generated 251 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:57:35.510469 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea4ad7-46b7-427e-b117-319209db81f7 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:35.511157 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea4ad7-46b7-427e-b117-319209db81f7 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:35.511253 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea4ad7-46b7-427e-b117-319209db81f7 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:35.513052 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 598/1206] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:57:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:57:35.519613 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3b237-9b79-42e5-978d-c6dfc8d6eb0e None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:35.520182 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3b237-9b79-42e5-978d-c6dfc8d6eb0e None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:35.520182 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3b237-9b79-42e5-978d-c6dfc8d6eb0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:35.542103 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-b1e3b237-9b79-42e5-978d-c6dfc8d6eb0e None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:57:35.543531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-b1e3b237-9b79-42e5-978d-c6dfc8d6eb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:57:35.631284 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 609/1207] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:57:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:57:35.942883 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea4ad7-46b7-427e-b117-319209db81f7 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:36.064238 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea4ad7-46b7-427e-b117-319209db81f7 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:36.102687 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60db570-6d57-408f-9b15-efe0ec08f2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:36.103349 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60db570-6d57-408f-9b15-efe0ec08f2ef None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:36.103629 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60db570-6d57-408f-9b15-efe0ec08f2ef None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:36.104469 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60db570-6d57-408f-9b15-efe0ec08f2ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:36.116635 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60db570-6d57-408f-9b15-efe0ec08f2ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:57:36.122032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-e60db570-6d57-408f-9b15-efe0ec08f2ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:57:36.122032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:36.122032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:36.122032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:36.122032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:36.122032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:36.122032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:36.122032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:36.122032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:36.122032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:36.122032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:36.122032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:36.122032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:57:36.122032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:57:36.122032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:57:36.122032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:57:36.123223 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:57:36.123223 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:57:36.123223 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:57:36.123223 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:57:36.123223 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:36.123223 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:36.123223 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:36.123223 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:36.123223 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:36.123223 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:36.123223 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:36.123223 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:36.123223 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:36.123223 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:36.123223 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:36.123223 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:36.124104 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:36.124104 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:36.124104 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:36.124104 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:36.124104 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:57:36.124104 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:57:36.124104 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:57:36.124104 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:57:36.124104 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:57:36.124104 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:57:36.124104 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:57:36.124104 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:57:36.124104 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 599/1208] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 14:57:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:57:36.129525 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e3b237-9b79-42e5-978d-c6dfc8d6eb0e None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:36.180201 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:36.256250 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e3b237-9b79-42e5-978d-c6dfc8d6eb0e None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:36.290732 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef75e95-8bac-44fc-b0ca-5edafd2557de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:36.291898 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef75e95-8bac-44fc-b0ca-5edafd2557de None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:36.292393 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef75e95-8bac-44fc-b0ca-5edafd2557de None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:36.292815 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef75e95-8bac-44fc-b0ca-5edafd2557de None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:36.293411 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef75e95-8bac-44fc-b0ca-5edafd2557de None admin] RBAC: Adding query filter params (name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:36.293719 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef75e95-8bac-44fc-b0ca-5edafd2557de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:36.295136 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef75e95-8bac-44fc-b0ca-5edafd2557de None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:36.303424 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 610/1209] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 14:57:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:36.310385 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60db570-6d57-408f-9b15-efe0ec08f2ef None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:36.367257 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:36.423688 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60db570-6d57-408f-9b15-efe0ec08f2ef None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:36.461696 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bb1efd-cee9-4e4c-ad80-2e1182bb75bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:36.462348 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb1efd-cee9-4e4c-ad80-2e1182bb75bd None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:36.462620 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb1efd-cee9-4e4c-ad80-2e1182bb75bd None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:36.462943 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb1efd-cee9-4e4c-ad80-2e1182bb75bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:36.478566 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bb1efd-cee9-4e4c-ad80-2e1182bb75bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:36.480160 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bb1efd-cee9-4e4c-ad80-2e1182bb75bd None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:36.498884 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-46bb1efd-cee9-4e4c-ad80-2e1182bb75bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:57:36.498884 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:36.498884 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:36.498884 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:36.498884 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:36.498884 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:36.498884 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:36.498884 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:36.498884 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:36.498884 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:36.498884 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:36.498884 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:36.498884 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:57:36.498884 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:57:36.498884 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:57:36.498884 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:57:36.500141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:36.500141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:36.500141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:36.500141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:36.500141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:36.500141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:36.500141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:36.500141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:36.500141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:36.500141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:36.500141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:36.500141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:36.500141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:36.500141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:36.500141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:36.500141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:36.500141 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:57:36.501350 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:57:36.501350 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:57:36.501350 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:57:36.505188 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 600/1210] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:57:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:57:36.506488 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef75e95-8bac-44fc-b0ca-5edafd2557de None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:36.628064 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:36.632666 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef75e95-8bac-44fc-b0ca-5edafd2557de None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:36.683148 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a85e17-2010-49d1-a751-719c8f4d57f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:36.683767 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a85e17-2010-49d1-a751-719c8f4d57f7 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:36.684054 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a85e17-2010-49d1-a751-719c8f4d57f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:36.684343 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a85e17-2010-49d1-a751-719c8f4d57f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:36.685353 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a85e17-2010-49d1-a751-719c8f4d57f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:36.685625 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a85e17-2010-49d1-a751-719c8f4d57f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:36.687313 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a85e17-2010-49d1-a751-719c8f4d57f7 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:36.700581 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 611/1211] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:57:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:57:36.705995 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bb1efd-cee9-4e4c-ad80-2e1182bb75bd None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:36.820948 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bb1efd-cee9-4e4c-ad80-2e1182bb75bd None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:36.822687 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:36.861086 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96187e-7004-4398-a8ee-70938e24c3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:36.861731 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96187e-7004-4398-a8ee-70938e24c3cf None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:36.862011 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96187e-7004-4398-a8ee-70938e24c3cf None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:36.862284 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96187e-7004-4398-a8ee-70938e24c3cf None admin] PATH_INFO: `/v3/users/placement` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:36.871487 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96187e-7004-4398-a8ee-70938e24c3cf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:36.873110 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96187e-7004-4398-a8ee-70938e24c3cf None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:36.884870 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-cb96187e-7004-4398-a8ee-70938e24c3cf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 14:57:36.884870 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:36.884870 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:36.884870 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:36.884870 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:36.884870 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:36.884870 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:36.884870 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:36.884870 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:36.884870 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:36.884870 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:36.884870 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:36.884870 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:57:36.884870 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:57:36.884870 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:57:36.884870 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:57:36.886095 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:36.886095 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:36.886095 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:57:36.886095 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:57:36.886095 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:57:36.886095 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:57:36.886095 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:36.886095 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:36.886095 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:36.886095 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:36.886095 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:36.886095 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:36.886095 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:36.886095 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:36.886095 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:36.886095 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:36.886095 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:36.886095 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:36.887150 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:36.887150 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:36.887150 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:57:36.887150 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:57:36.887150 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:57:36.887150 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:57:36.887150 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:57:36.887150 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:57:36.887150 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 14:57:36.887150 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:57:36.887150 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 601/1212] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:57:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:57:36.892058 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a85e17-2010-49d1-a751-719c8f4d57f7 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:37.017311 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a85e17-2010-49d1-a751-719c8f4d57f7 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:37.054037 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628fa0de-7918-4d7c-8b69-166025d91b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:37.054872 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628fa0de-7918-4d7c-8b69-166025d91b71 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:37.055253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628fa0de-7918-4d7c-8b69-166025d91b71 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:37.055639 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628fa0de-7918-4d7c-8b69-166025d91b71 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:37.056456 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628fa0de-7918-4d7c-8b69-166025d91b71 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:37.056898 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628fa0de-7918-4d7c-8b69-166025d91b71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:37.058445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628fa0de-7918-4d7c-8b69-166025d91b71 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:37.083224 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 612/1213] 10.222.0.33 () {56 vars in 1143 bytes} [Sun Sep 11 14:57:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:57:37.084140 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96187e-7004-4398-a8ee-70938e24c3cf None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:37.197937 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96187e-7004-4398-a8ee-70938e24c3cf None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:37.236061 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24644c8-c879-4b7d-ad73-6c5e767a6442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:37.236695 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24644c8-c879-4b7d-ad73-6c5e767a6442 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:37.236978 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24644c8-c879-4b7d-ad73-6c5e767a6442 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:37.237254 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24644c8-c879-4b7d-ad73-6c5e767a6442 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:37.267852 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:37.316057 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24644c8-c879-4b7d-ad73-6c5e767a6442 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:37.317574 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24644c8-c879-4b7d-ad73-6c5e767a6442 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:37.432755 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 602/1214] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:57:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 14:57:37.439347 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628fa0de-7918-4d7c-8b69-166025d91b71 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:37.463830 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:37.560482 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628fa0de-7918-4d7c-8b69-166025d91b71 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:37.604776 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b398e6-8324-4f2b-9852-f9c988204b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:37.604776 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b398e6-8324-4f2b-9852-f9c988204b55 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:37.604776 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b398e6-8324-4f2b-9852-f9c988204b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:37.605133 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b398e6-8324-4f2b-9852-f9c988204b55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:37.618162 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b398e6-8324-4f2b-9852-f9c988204b55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:37.619877 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b398e6-8324-4f2b-9852-f9c988204b55 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:37.636876 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-d7b398e6-8324-4f2b-9852-f9c988204b55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:57:37.636876 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:37.636876 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:37.636876 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:37.636876 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:37.636876 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:37.636876 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:37.636876 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:37.636876 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:37.636876 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:37.636876 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:37.636876 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:37.636876 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:57:37.636876 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:57:37.636876 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:57:37.636876 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:57:37.638736 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:37.638736 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:37.638736 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:37.638736 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:37.638736 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:37.638736 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:37.638736 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:37.638736 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:37.638736 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:37.638736 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:37.638736 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:37.638736 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:37.638736 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:37.638736 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:37.638736 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:37.638736 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:37.638736 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:57:37.639717 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:57:37.639717 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:57:37.639717 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:57:37.639717 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 613/1215] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:57:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:57:37.644319 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24644c8-c879-4b7d-ad73-6c5e767a6442 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:37.715103 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:37.757602 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24644c8-c879-4b7d-ad73-6c5e767a6442 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:37.802150 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e31b6-3e04-413e-ad70-10546e5e8ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:37.802820 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e31b6-3e04-413e-ad70-10546e5e8ddf None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:37.803171 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e31b6-3e04-413e-ad70-10546e5e8ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:37.803528 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e31b6-3e04-413e-ad70-10546e5e8ddf None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:37.804327 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928e31b6-3e04-413e-ad70-10546e5e8ddf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:37.804627 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928e31b6-3e04-413e-ad70-10546e5e8ddf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:37.806081 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928e31b6-3e04-413e-ad70-10546e5e8ddf None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:37.818257 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 603/1216] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:57:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:37.825229 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b398e6-8324-4f2b-9852-f9c988204b55 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:37.905874 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:37.942310 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b398e6-8324-4f2b-9852-f9c988204b55 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:37.982979 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eed9f6-5c7d-4b81-bdad-f6d1a470979f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:37.983713 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eed9f6-5c7d-4b81-bdad-f6d1a470979f None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:37.984080 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eed9f6-5c7d-4b81-bdad-f6d1a470979f None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:37.984281 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eed9f6-5c7d-4b81-bdad-f6d1a470979f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:37.992457 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eed9f6-5c7d-4b81-bdad-f6d1a470979f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:37.994167 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eed9f6-5c7d-4b81-bdad-f6d1a470979f None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:38.008457 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-d9eed9f6-5c7d-4b81-bdad-f6d1a470979f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:57:38.008457 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:38.008457 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:38.008457 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:38.008457 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:38.008457 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:38.008457 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:38.008457 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:38.008457 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:38.008457 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:38.008457 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:38.008457 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:38.008457 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:57:38.008457 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:57:38.008457 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:57:38.009620 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:57:38.009620 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:38.009620 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:38.009620 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:38.009620 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:38.009620 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:38.009620 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:38.009620 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:38.009620 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:38.009620 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:38.009620 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:38.009620 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:38.009620 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:38.009620 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:38.009620 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:38.009620 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:38.009620 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:38.011196 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:57:38.011196 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:57:38.011196 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:57:38.011196 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:57:38.011196 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:57:38.011196 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:57:38.011196 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:57:38.011196 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:57:38.015029 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 614/1217] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:57:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:57:38.016976 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e31b6-3e04-413e-ad70-10546e5e8ddf None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:38.137914 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e31b6-3e04-413e-ad70-10546e5e8ddf None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:38.163500 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:38.176086 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea783e1e-97a8-4a1c-9bed-16aa83a78067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:38.176701 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea783e1e-97a8-4a1c-9bed-16aa83a78067 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:38.176987 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea783e1e-97a8-4a1c-9bed-16aa83a78067 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:38.177287 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea783e1e-97a8-4a1c-9bed-16aa83a78067 None admin] PATH_INFO: `/v3/projects` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:38.177972 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea783e1e-97a8-4a1c-9bed-16aa83a78067 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:38.178278 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea783e1e-97a8-4a1c-9bed-16aa83a78067 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:38.179753 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea783e1e-97a8-4a1c-9bed-16aa83a78067 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:38.193960 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 604/1218] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:57:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:57:38.201396 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eed9f6-5c7d-4b81-bdad-f6d1a470979f None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:38.309442 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eed9f6-5c7d-4b81-bdad-f6d1a470979f None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:38.338594 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:38.345124 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e9c27-3e2c-4e44-b75b-aa268c49d3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:38.345743 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e9c27-3e2c-4e44-b75b-aa268c49d3d7 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:38.346040 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e9c27-3e2c-4e44-b75b-aa268c49d3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:38.346311 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e9c27-3e2c-4e44-b75b-aa268c49d3d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:38.347081 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323e9c27-3e2c-4e44-b75b-aa268c49d3d7 None admin] RBAC: Adding query filter params (role.id=533fa341c4f64cd4b0a7ebb668f9106a, scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176, user.id=9390c5960b404bedbead2becfdc6c7af) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:38.347391 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323e9c27-3e2c-4e44-b75b-aa268c49d3d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:38.348890 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323e9c27-3e2c-4e44-b75b-aa268c49d3d7 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:38.355400 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 615/1219] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:57:38 2022] GET /identity/v3/role_assignments?user.id=9390c5960b404bedbead2becfdc6c7af&scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176&role.id=533fa341c4f64cd4b0a7ebb668f9106a => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:40.593780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126777c-746d-4b46-ade7-372da5bd187b None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:40.594617 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126777c-746d-4b46-ade7-372da5bd187b None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:40.594617 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126777c-746d-4b46-ade7-372da5bd187b None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:40.596177 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 605/1220] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:57:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:57:40.602105 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588625e7-a47a-4c10-8969-c72f311ef6a4 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:40.602629 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588625e7-a47a-4c10-8969-c72f311ef6a4 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:40.602741 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588625e7-a47a-4c10-8969-c72f311ef6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:40.622635 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-588625e7-a47a-4c10-8969-c72f311ef6a4 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:57:40.624036 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-588625e7-a47a-4c10-8969-c72f311ef6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:57:40.701462 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 616/1221] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:57:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:57:40.993337 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126777c-746d-4b46-ade7-372da5bd187b None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:41.112257 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126777c-746d-4b46-ade7-372da5bd187b None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:41.150161 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28e03a-8662-4e44-a02d-563306b6d425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:41.151250 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28e03a-8662-4e44-a02d-563306b6d425 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:41.151555 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28e03a-8662-4e44-a02d-563306b6d425 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:41.151864 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28e03a-8662-4e44-a02d-563306b6d425 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:41.163206 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e28e03a-8662-4e44-a02d-563306b6d425 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:57:41.169173 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-5e28e03a-8662-4e44-a02d-563306b6d425 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:57:41.169173 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:41.169173 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:41.169173 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:41.169173 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:41.169173 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:41.169173 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:41.169173 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:41.169173 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:41.169173 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:41.169173 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:41.169173 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:41.169173 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:57:41.169173 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:57:41.169173 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:57:41.169173 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:57:41.170370 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:57:41.170370 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:57:41.170370 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:57:41.170370 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:57:41.170370 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:41.170370 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:41.170370 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:41.170370 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:41.170370 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:41.170370 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:41.170370 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:41.170370 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:41.170370 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:41.170370 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:41.170370 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:41.170370 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:41.171284 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:41.171284 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:41.171284 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:41.171284 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:41.171284 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:57:41.171284 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:57:41.171284 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:57:41.171284 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:57:41.171284 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:57:41.171284 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:57:41.171284 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:57:41.171284 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:57:41.171284 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 606/1222] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 14:57:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:57:41.176902 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588625e7-a47a-4c10-8969-c72f311ef6a4 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:41.283637 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588625e7-a47a-4c10-8969-c72f311ef6a4 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:41.309928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:41.324015 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ecf42a-84f9-477a-ab2b-be8d8448653f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:41.324698 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecf42a-84f9-477a-ab2b-be8d8448653f None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:41.324977 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecf42a-84f9-477a-ab2b-be8d8448653f None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:41.325250 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecf42a-84f9-477a-ab2b-be8d8448653f None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:41.325988 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ecf42a-84f9-477a-ab2b-be8d8448653f None admin] RBAC: Adding query filter params (name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:41.326295 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ecf42a-84f9-477a-ab2b-be8d8448653f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:41.327815 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ecf42a-84f9-477a-ab2b-be8d8448653f None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:41.336652 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 617/1223] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 14:57:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:41.343593 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28e03a-8662-4e44-a02d-563306b6d425 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:41.396902 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:41.467416 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28e03a-8662-4e44-a02d-563306b6d425 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:41.503963 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6adb963-4e9f-40b1-ab35-94c062dbdcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:41.504597 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6adb963-4e9f-40b1-ab35-94c062dbdcbe None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:41.504876 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6adb963-4e9f-40b1-ab35-94c062dbdcbe None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:41.505147 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6adb963-4e9f-40b1-ab35-94c062dbdcbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:41.515459 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6adb963-4e9f-40b1-ab35-94c062dbdcbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:41.516915 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6adb963-4e9f-40b1-ab35-94c062dbdcbe None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:41.532211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-b6adb963-4e9f-40b1-ab35-94c062dbdcbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:57:41.532211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:41.532211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:41.532211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:41.532211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:41.532211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:41.532211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:41.532211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:41.532211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:41.532211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:41.532211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:41.532211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:41.532211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:57:41.532211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:57:41.532211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:57:41.532211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:57:41.533965 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:41.533965 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:41.533965 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:41.533965 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:41.533965 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:41.533965 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:41.533965 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:41.533965 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:41.533965 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:41.533965 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:41.533965 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:41.533965 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:41.533965 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:41.533965 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:41.533965 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:41.533965 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:41.533965 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:57:41.535003 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:57:41.535003 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:57:41.535003 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:57:41.535003 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 607/1224] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:57:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:57:41.539175 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ecf42a-84f9-477a-ab2b-be8d8448653f None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:41.661943 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ecf42a-84f9-477a-ab2b-be8d8448653f None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:41.698834 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49675487-15da-4709-91ee-115514a05c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:41.699456 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49675487-15da-4709-91ee-115514a05c91 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:41.699738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49675487-15da-4709-91ee-115514a05c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:41.700014 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49675487-15da-4709-91ee-115514a05c91 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:41.700662 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49675487-15da-4709-91ee-115514a05c91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:41.700969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49675487-15da-4709-91ee-115514a05c91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:41.702359 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49675487-15da-4709-91ee-115514a05c91 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:41.715388 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 618/1225] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:57:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:41.721146 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6adb963-4e9f-40b1-ab35-94c062dbdcbe None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:41.820742 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:41.828497 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6adb963-4e9f-40b1-ab35-94c062dbdcbe None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:41.869728 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de94ef6-a817-45b6-8df8-d73be9037756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:41.870716 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de94ef6-a817-45b6-8df8-d73be9037756 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:41.871245 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de94ef6-a817-45b6-8df8-d73be9037756 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:41.871734 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de94ef6-a817-45b6-8df8-d73be9037756 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:41.879963 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de94ef6-a817-45b6-8df8-d73be9037756 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:41.881430 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de94ef6-a817-45b6-8df8-d73be9037756 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:41.894080 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-4de94ef6-a817-45b6-8df8-d73be9037756 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 14:57:41.894080 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:41.894080 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:41.894080 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:41.894080 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:41.894080 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:41.894080 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:41.894080 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:41.894080 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:41.894080 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:41.894080 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:41.894080 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:41.894080 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:57:41.894080 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:57:41.894080 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:57:41.894080 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:57:41.895366 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:41.895366 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:41.895366 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:57:41.895366 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:57:41.895366 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:57:41.895366 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:57:41.895366 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:41.895366 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:41.895366 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:41.895366 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:41.895366 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:41.895366 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:41.895366 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:41.895366 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:41.895366 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:41.895366 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:41.895366 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:41.895366 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:41.896661 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:41.896661 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:41.896661 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:57:41.896661 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:57:41.896661 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:57:41.896661 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:57:41.896661 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:57:41.896661 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:57:41.896661 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 14:57:41.896661 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:57:41.896661 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 608/1226] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:57:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:57:41.901362 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49675487-15da-4709-91ee-115514a05c91 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:41.954734 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:42.016403 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49675487-15da-4709-91ee-115514a05c91 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:42.053036 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb7f72-8581-42f9-bee8-6aed518a4d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:42.053640 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb7f72-8581-42f9-bee8-6aed518a4d91 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:42.053918 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb7f72-8581-42f9-bee8-6aed518a4d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:42.054189 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb7f72-8581-42f9-bee8-6aed518a4d91 None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:42.054945 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb7f72-8581-42f9-bee8-6aed518a4d91 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:42.055251 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb7f72-8581-42f9-bee8-6aed518a4d91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:42.057490 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb7f72-8581-42f9-bee8-6aed518a4d91 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:42.076141 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 619/1227] 10.222.0.33 () {56 vars in 1143 bytes} [Sun Sep 11 14:57:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:42.081885 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de94ef6-a817-45b6-8df8-d73be9037756 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:42.196437 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de94ef6-a817-45b6-8df8-d73be9037756 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:42.231869 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f859e49-bcd4-4160-a6d0-3527e4925603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:42.232497 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f859e49-bcd4-4160-a6d0-3527e4925603 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:42.232778 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f859e49-bcd4-4160-a6d0-3527e4925603 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:42.233052 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f859e49-bcd4-4160-a6d0-3527e4925603 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:42.245073 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f859e49-bcd4-4160-a6d0-3527e4925603 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:42.246716 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f859e49-bcd4-4160-a6d0-3527e4925603 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:42.263617 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-1f859e49-bcd4-4160-a6d0-3527e4925603 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:57:42.263617 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:42.263617 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:42.263617 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:42.263617 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:42.263617 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:42.263617 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:42.263617 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:42.263617 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:42.263617 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:42.263617 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:42.263617 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:42.263617 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:57:42.263617 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:57:42.263617 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:57:42.263617 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:57:42.264945 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:42.264945 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:42.264945 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:42.264945 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:42.264945 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:42.264945 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:42.264945 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:42.264945 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:42.264945 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:42.264945 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:42.264945 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:42.264945 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:42.264945 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:42.264945 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:42.264945 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:42.264945 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:42.264945 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:57:42.265942 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:57:42.265942 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:57:42.265942 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:57:42.265942 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 609/1228] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:57:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:57:42.270938 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb7f72-8581-42f9-bee8-6aed518a4d91 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:42.379552 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb7f72-8581-42f9-bee8-6aed518a4d91 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:42.381792 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:42.417902 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0477c9f-b4a1-4f01-8574-624e3d03ddcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:42.419010 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0477c9f-b4a1-4f01-8574-624e3d03ddcd None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:42.419375 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0477c9f-b4a1-4f01-8574-624e3d03ddcd None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:42.419684 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0477c9f-b4a1-4f01-8574-624e3d03ddcd None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:42.420465 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0477c9f-b4a1-4f01-8574-624e3d03ddcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:42.420791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0477c9f-b4a1-4f01-8574-624e3d03ddcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:42.422431 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0477c9f-b4a1-4f01-8574-624e3d03ddcd None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:42.436355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 620/1229] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:57:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:42.442151 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f859e49-bcd4-4160-a6d0-3527e4925603 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:42.453334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:42.555432 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f859e49-bcd4-4160-a6d0-3527e4925603 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:42.598912 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d539f-3b93-4a07-8c72-51c51c54ec76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:42.599550 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d539f-3b93-4a07-8c72-51c51c54ec76 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:42.599828 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d539f-3b93-4a07-8c72-51c51c54ec76 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:42.600136 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d539f-3b93-4a07-8c72-51c51c54ec76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:42.607577 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d539f-3b93-4a07-8c72-51c51c54ec76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:42.609469 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d539f-3b93-4a07-8c72-51c51c54ec76 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:42.622990 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-087d539f-3b93-4a07-8c72-51c51c54ec76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:57:42.622990 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:42.622990 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:42.622990 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:42.622990 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:42.622990 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:42.622990 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:42.622990 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:42.622990 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:42.622990 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:42.622990 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:42.622990 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:42.622990 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:57:42.622990 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:57:42.622990 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:57:42.624215 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:57:42.624215 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:42.624215 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:42.624215 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:42.624215 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:42.624215 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:42.624215 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:42.624215 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:42.624215 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:42.624215 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:42.624215 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:42.624215 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:42.624215 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:42.624215 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:42.624215 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:42.624215 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:42.624215 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:42.625280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:57:42.625280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:57:42.625280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:57:42.625280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:57:42.625280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:57:42.625280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:57:42.625280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:57:42.625280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:57:42.625280 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 610/1230] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:57:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:57:42.630879 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0477c9f-b4a1-4f01-8574-624e3d03ddcd None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:42.737290 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0477c9f-b4a1-4f01-8574-624e3d03ddcd None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:42.771754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b858b75f-48a6-4b44-9dbc-3133eea2f968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:42.772335 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858b75f-48a6-4b44-9dbc-3133eea2f968 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:42.772625 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858b75f-48a6-4b44-9dbc-3133eea2f968 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:42.772915 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858b75f-48a6-4b44-9dbc-3133eea2f968 None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:42.773653 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b858b75f-48a6-4b44-9dbc-3133eea2f968 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:42.773960 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b858b75f-48a6-4b44-9dbc-3133eea2f968 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:42.776075 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b858b75f-48a6-4b44-9dbc-3133eea2f968 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:42.789551 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 621/1231] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:57:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:42.796069 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d539f-3b93-4a07-8c72-51c51c54ec76 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:42.868502 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:42.910429 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d539f-3b93-4a07-8c72-51c51c54ec76 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:42.949805 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801ffe75-325e-49e2-b1e8-562a38bd7082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:42.950470 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ffe75-325e-49e2-b1e8-562a38bd7082 None admin] REQUEST_METHOD: `PUT` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:42.950745 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ffe75-325e-49e2-b1e8-562a38bd7082 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:42.951053 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ffe75-325e-49e2-b1e8-562a38bd7082 None admin] PATH_INFO: `/v3/projects/9e4a3d44808e4b4092cfc4ee284c0176/users/9390c5960b404bedbead2becfdc6c7af/roles/533fa341c4f64cd4b0a7ebb668f9106a` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:42.967205 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801ffe75-325e-49e2-b1e8-562a38bd7082 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e4a3d44808e4b4092cfc4ee284c0176, user_id=9390c5960b404bedbead2becfdc6c7af, role_id=533fa341c4f64cd4b0a7ebb668f9106a)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:42.968669 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801ffe75-325e-49e2-b1e8-562a38bd7082 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:42.983027 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 611/1232] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 14:57:42 2022] PUT /identity/v3/projects/9e4a3d44808e4b4092cfc4ee284c0176/users/9390c5960b404bedbead2becfdc6c7af/roles/533fa341c4f64cd4b0a7ebb668f9106a => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:57:45.185731 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415d839-52f5-4fa4-9f95-abc1dae98a7c None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:45.186291 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415d839-52f5-4fa4-9f95-abc1dae98a7c None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:45.186291 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415d839-52f5-4fa4-9f95-abc1dae98a7c None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:45.187642 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 622/1233] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:57:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:57:45.192736 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48b16d-6a40-44b7-8de5-5485b62263b4 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:45.193185 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48b16d-6a40-44b7-8de5-5485b62263b4 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:45.193288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48b16d-6a40-44b7-8de5-5485b62263b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:45.212982 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-8f48b16d-6a40-44b7-8de5-5485b62263b4 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:57:45.214677 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-8f48b16d-6a40-44b7-8de5-5485b62263b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:57:45.285702 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 612/1234] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:57:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:57:45.564168 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7415d839-52f5-4fa4-9f95-abc1dae98a7c None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:45.592995 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:45.678558 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7415d839-52f5-4fa4-9f95-abc1dae98a7c None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:45.715063 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c336c6bb-e7a1-43b4-98d5-373a484fd3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:45.715723 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336c6bb-e7a1-43b4-98d5-373a484fd3a8 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:45.715972 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336c6bb-e7a1-43b4-98d5-373a484fd3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:45.716235 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336c6bb-e7a1-43b4-98d5-373a484fd3a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:45.726230 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c336c6bb-e7a1-43b4-98d5-373a484fd3a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:57:45.731246 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-c336c6bb-e7a1-43b4-98d5-373a484fd3a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:57:45.731246 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:45.731246 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:45.731246 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:45.731246 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:45.731246 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:45.731246 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:45.731246 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:45.731246 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:45.731246 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:45.731246 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:45.731246 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:45.731246 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:57:45.731246 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:57:45.731246 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:57:45.731246 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:57:45.734880 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:57:45.734880 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:57:45.734880 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:57:45.734880 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:57:45.734880 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:45.734880 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:45.734880 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:45.734880 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:45.734880 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:45.734880 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:45.734880 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:45.734880 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:45.734880 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:45.734880 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:45.734880 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:45.734880 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:45.735961 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:45.735961 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:45.735961 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:45.735961 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:45.735961 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:57:45.735961 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:57:45.735961 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:57:45.735961 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:57:45.735961 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:57:45.735961 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:57:45.735961 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:57:45.735961 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:57:45.735961 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 623/1235] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 14:57:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:57:45.739423 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f48b16d-6a40-44b7-8de5-5485b62263b4 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:45.787882 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:45.846390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f48b16d-6a40-44b7-8de5-5485b62263b4 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:45.881131 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9db78-3aff-45eb-95ee-ddf8bb81dab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:45.881788 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9db78-3aff-45eb-95ee-ddf8bb81dab2 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:45.882068 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9db78-3aff-45eb-95ee-ddf8bb81dab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:45.882334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9db78-3aff-45eb-95ee-ddf8bb81dab2 None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:45.883016 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f9db78-3aff-45eb-95ee-ddf8bb81dab2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:45.883314 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f9db78-3aff-45eb-95ee-ddf8bb81dab2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:45.884673 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f9db78-3aff-45eb-95ee-ddf8bb81dab2 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:45.893061 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 613/1236] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 14:57:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:45.898948 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c336c6bb-e7a1-43b4-98d5-373a484fd3a8 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:45.993070 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:46.011380 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c336c6bb-e7a1-43b4-98d5-373a484fd3a8 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:46.050711 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e2dee7-1708-4b9a-b25d-105885850f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:46.051409 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e2dee7-1708-4b9a-b25d-105885850f76 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:46.051706 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e2dee7-1708-4b9a-b25d-105885850f76 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:46.052027 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e2dee7-1708-4b9a-b25d-105885850f76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:46.065247 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e2dee7-1708-4b9a-b25d-105885850f76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:46.066713 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e2dee7-1708-4b9a-b25d-105885850f76 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:46.082707 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-40e2dee7-1708-4b9a-b25d-105885850f76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:57:46.082707 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:46.082707 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:46.082707 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:46.082707 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:46.082707 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:46.082707 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:46.082707 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:46.082707 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:46.082707 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:46.082707 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:46.082707 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:46.082707 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:57:46.082707 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:57:46.082707 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:57:46.082707 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:57:46.083914 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:46.083914 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:46.083914 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:46.083914 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:46.083914 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:46.083914 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:46.083914 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:46.083914 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:46.083914 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:46.083914 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:46.083914 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:46.083914 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:46.083914 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:46.083914 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:46.083914 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:46.083914 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:46.083914 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:57:46.084985 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:57:46.084985 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:57:46.084985 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:57:46.084985 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 624/1237] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:57:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:57:46.090450 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9db78-3aff-45eb-95ee-ddf8bb81dab2 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:46.199571 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9db78-3aff-45eb-95ee-ddf8bb81dab2 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:46.209363 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:46.236713 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15464fd7-ff6e-49c1-8ab8-d260c301b16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:46.237318 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15464fd7-ff6e-49c1-8ab8-d260c301b16f None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:46.237595 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15464fd7-ff6e-49c1-8ab8-d260c301b16f None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:46.237864 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15464fd7-ff6e-49c1-8ab8-d260c301b16f None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:46.238492 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15464fd7-ff6e-49c1-8ab8-d260c301b16f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:46.238788 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15464fd7-ff6e-49c1-8ab8-d260c301b16f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:46.240176 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15464fd7-ff6e-49c1-8ab8-d260c301b16f None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:46.252727 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 614/1238] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:57:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:46.258574 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e2dee7-1708-4b9a-b25d-105885850f76 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:46.365882 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e2dee7-1708-4b9a-b25d-105885850f76 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:46.403733 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b109d-a06a-4ab9-8208-6a33569f3d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:46.404357 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b109d-a06a-4ab9-8208-6a33569f3d81 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:46.404633 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b109d-a06a-4ab9-8208-6a33569f3d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:46.404903 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b109d-a06a-4ab9-8208-6a33569f3d81 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:46.407594 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:46.413661 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b109d-a06a-4ab9-8208-6a33569f3d81 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:46.415621 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b109d-a06a-4ab9-8208-6a33569f3d81 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:46.428213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-fe8b109d-a06a-4ab9-8208-6a33569f3d81 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 14:57:46.428213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:46.428213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:46.428213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:46.428213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:46.428213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:46.428213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:46.428213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:46.428213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:46.428213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:46.428213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:46.428213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:46.428213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:57:46.428213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:57:46.428213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:57:46.428213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:57:46.429535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:46.429535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:46.429535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:57:46.429535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:57:46.429535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:57:46.429535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:57:46.429535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:46.429535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:46.429535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:46.429535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:46.429535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:46.429535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:46.429535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:46.429535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:46.429535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:46.429535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:46.429535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:46.429535 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:46.430518 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:46.430518 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:46.430518 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:57:46.430518 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:57:46.430518 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:57:46.430518 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:57:46.430518 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:57:46.430518 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:57:46.430518 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 14:57:46.430518 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:57:46.430518 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 625/1239] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:57:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:57:46.435745 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15464fd7-ff6e-49c1-8ab8-d260c301b16f None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:46.541858 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15464fd7-ff6e-49c1-8ab8-d260c301b16f None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:46.578973 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc69b8-0bcd-4ba9-929a-5349618e9fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:46.579588 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc69b8-0bcd-4ba9-929a-5349618e9fd3 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:46.579857 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc69b8-0bcd-4ba9-929a-5349618e9fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:46.580133 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc69b8-0bcd-4ba9-929a-5349618e9fd3 None admin] PATH_INFO: `/v3/users` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:46.580866 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc69b8-0bcd-4ba9-929a-5349618e9fd3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:46.581168 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc69b8-0bcd-4ba9-929a-5349618e9fd3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:46.582613 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc69b8-0bcd-4ba9-929a-5349618e9fd3 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:46.605167 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 615/1240] 10.222.0.33 () {56 vars in 1143 bytes} [Sun Sep 11 14:57:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:46.611416 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b109d-a06a-4ab9-8208-6a33569f3d81 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:46.729520 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b109d-a06a-4ab9-8208-6a33569f3d81 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:46.767978 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15236932-dc4a-4ddf-8fcd-6683ced93ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:46.768744 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15236932-dc4a-4ddf-8fcd-6683ced93ee6 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:46.769052 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15236932-dc4a-4ddf-8fcd-6683ced93ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:46.769319 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15236932-dc4a-4ddf-8fcd-6683ced93ee6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:46.841851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:46.849708 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15236932-dc4a-4ddf-8fcd-6683ced93ee6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:46.851207 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15236932-dc4a-4ddf-8fcd-6683ced93ee6 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:46.963908 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 626/1241] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:57:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 14:57:46.970110 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc69b8-0bcd-4ba9-929a-5349618e9fd3 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:47.001413 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:47.082693 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc69b8-0bcd-4ba9-929a-5349618e9fd3 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:47.121842 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5536daba-99d7-4d35-b7db-4c4537090193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:47.122441 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5536daba-99d7-4d35-b7db-4c4537090193 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:47.122766 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5536daba-99d7-4d35-b7db-4c4537090193 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:47.123101 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5536daba-99d7-4d35-b7db-4c4537090193 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:47.133949 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5536daba-99d7-4d35-b7db-4c4537090193 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:47.136170 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5536daba-99d7-4d35-b7db-4c4537090193 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:47.153444 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-5536daba-99d7-4d35-b7db-4c4537090193 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:57:47.153444 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:47.153444 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:47.153444 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:47.153444 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:47.153444 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:47.153444 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:47.153444 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:47.153444 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:47.153444 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:47.153444 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:47.153444 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:47.153444 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:57:47.153444 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:57:47.153444 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:57:47.153444 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:57:47.154666 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:47.154666 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:47.154666 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:47.154666 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:47.154666 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:47.154666 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:47.154666 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:47.154666 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:47.154666 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:47.154666 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:47.154666 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:47.154666 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:47.154666 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:47.154666 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:47.154666 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:47.154666 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:47.154666 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:57:47.155962 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:57:47.155962 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:57:47.155962 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:57:47.155962 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 616/1242] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:57:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:57:47.160519 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15236932-dc4a-4ddf-8fcd-6683ced93ee6 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:47.265688 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:47.272262 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15236932-dc4a-4ddf-8fcd-6683ced93ee6 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:47.309098 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0106e4dd-778f-44c6-a2d2-a8fd4133cc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:47.309702 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106e4dd-778f-44c6-a2d2-a8fd4133cc80 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:47.309979 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106e4dd-778f-44c6-a2d2-a8fd4133cc80 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:47.310489 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106e4dd-778f-44c6-a2d2-a8fd4133cc80 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:47.311152 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0106e4dd-778f-44c6-a2d2-a8fd4133cc80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:47.311450 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0106e4dd-778f-44c6-a2d2-a8fd4133cc80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:47.312846 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0106e4dd-778f-44c6-a2d2-a8fd4133cc80 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:47.324754 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 627/1243] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:57:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:47.330747 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5536daba-99d7-4d35-b7db-4c4537090193 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:47.414852 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:47.440383 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5536daba-99d7-4d35-b7db-4c4537090193 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:47.477277 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad046e7-fdbb-465f-a4bf-0f9bb9b1e8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:47.480109 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad046e7-fdbb-465f-a4bf-0f9bb9b1e8dd None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:47.480109 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad046e7-fdbb-465f-a4bf-0f9bb9b1e8dd None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:47.480109 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad046e7-fdbb-465f-a4bf-0f9bb9b1e8dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:47.488438 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad046e7-fdbb-465f-a4bf-0f9bb9b1e8dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:47.489905 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad046e7-fdbb-465f-a4bf-0f9bb9b1e8dd None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:47.505057 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-cad046e7-fdbb-465f-a4bf-0f9bb9b1e8dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:57:47.505057 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:47.505057 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:47.505057 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:47.505057 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:47.505057 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:47.505057 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:47.505057 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:47.505057 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:47.505057 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:47.505057 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:47.505057 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:47.505057 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:57:47.505057 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:57:47.505057 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:57:47.506094 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:57:47.506094 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:47.506094 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:47.506094 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:47.506094 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:47.506094 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:47.506094 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:47.506094 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:47.506094 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:47.506094 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:47.506094 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:47.506094 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:47.506094 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:47.506094 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:47.506094 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:47.506094 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:47.506094 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:47.511427 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:57:47.511427 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:57:47.511427 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:57:47.511427 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:57:47.511427 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:57:47.511427 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:57:47.511427 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:57:47.511427 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:57:47.511427 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 617/1244] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:57:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:57:47.513607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0106e4dd-778f-44c6-a2d2-a8fd4133cc80 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:47.645087 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0106e4dd-778f-44c6-a2d2-a8fd4133cc80 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:47.684328 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9fbad0-3b99-496a-91ae-067c9ed8d7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:47.684859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fbad0-3b99-496a-91ae-067c9ed8d7e6 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:47.685136 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fbad0-3b99-496a-91ae-067c9ed8d7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:47.685688 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fbad0-3b99-496a-91ae-067c9ed8d7e6 None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:47.686338 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9fbad0-3b99-496a-91ae-067c9ed8d7e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:47.686636 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9fbad0-3b99-496a-91ae-067c9ed8d7e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:47.688109 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9fbad0-3b99-496a-91ae-067c9ed8d7e6 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:47.701787 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 628/1245] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:57:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:47.708334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad046e7-fdbb-465f-a4bf-0f9bb9b1e8dd None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:47.821319 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad046e7-fdbb-465f-a4bf-0f9bb9b1e8dd None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:47.863096 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b767a-e432-426e-8e85-9b550ad343a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:47.863706 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b767a-e432-426e-8e85-9b550ad343a8 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:47.864005 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b767a-e432-426e-8e85-9b550ad343a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:47.864258 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b767a-e432-426e-8e85-9b550ad343a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:47.864978 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39b767a-e432-426e-8e85-9b550ad343a8 None admin] RBAC: Adding query filter params (role.id=533fa341c4f64cd4b0a7ebb668f9106a, scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176, user.id=9390c5960b404bedbead2becfdc6c7af) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:47.865283 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39b767a-e432-426e-8e85-9b550ad343a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:47.866722 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39b767a-e432-426e-8e85-9b550ad343a8 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:47.873835 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 618/1246] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:57:47 2022] GET /identity/v3/role_assignments?user.id=9390c5960b404bedbead2becfdc6c7af&scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176&role.id=533fa341c4f64cd4b0a7ebb668f9106a => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:50.118825 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581a088-581e-4a00-bf0f-46eb55668cbc None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:50.119459 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581a088-581e-4a00-bf0f-46eb55668cbc None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:50.119459 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581a088-581e-4a00-bf0f-46eb55668cbc None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:50.120779 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 629/1247] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:57:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:57:50.126674 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432dc8e-9fa1-499d-9e3c-2ad50e439092 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:50.127188 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432dc8e-9fa1-499d-9e3c-2ad50e439092 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:50.127477 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432dc8e-9fa1-499d-9e3c-2ad50e439092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:50.134043 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:50.146948 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-7432dc8e-9fa1-499d-9e3c-2ad50e439092 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:57:50.148146 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-7432dc8e-9fa1-499d-9e3c-2ad50e439092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:57:50.220304 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 619/1248] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:57:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:57:50.512961 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581a088-581e-4a00-bf0f-46eb55668cbc None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:50.532045 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:50.637028 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581a088-581e-4a00-bf0f-46eb55668cbc None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:50.675116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80eb71-9ae0-462f-b47f-a97a7bc6c27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:50.675747 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80eb71-9ae0-462f-b47f-a97a7bc6c27f None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:50.676031 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80eb71-9ae0-462f-b47f-a97a7bc6c27f None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:50.676301 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80eb71-9ae0-462f-b47f-a97a7bc6c27f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:50.689462 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e80eb71-9ae0-462f-b47f-a97a7bc6c27f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:57:50.695454 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-8e80eb71-9ae0-462f-b47f-a97a7bc6c27f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:57:50.695454 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:50.695454 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:50.695454 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:50.695454 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:50.695454 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:50.695454 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:50.695454 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:50.695454 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:50.695454 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:50.695454 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:50.695454 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:50.695454 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:57:50.695454 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:57:50.695454 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:57:50.695454 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:57:50.696599 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:57:50.696599 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:57:50.696599 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:57:50.696599 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:57:50.696599 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:50.696599 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:50.696599 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:50.696599 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:50.696599 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:50.696599 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:50.696599 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:50.696599 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:50.696599 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:50.696599 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:50.696599 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:50.696599 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:50.697638 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:50.697638 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:50.697638 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:50.697638 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:50.697638 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:57:50.697638 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:57:50.697638 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:57:50.697638 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:57:50.697638 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:57:50.697638 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:57:50.697638 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:57:50.697638 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:57:50.702622 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 630/1249] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:57:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:57:50.703629 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7432dc8e-9fa1-499d-9e3c-2ad50e439092 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:50.813128 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7432dc8e-9fa1-499d-9e3c-2ad50e439092 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:50.845330 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:50.850975 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588771fa-ef5e-4ae4-b577-dfbefd9606a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:50.851588 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588771fa-ef5e-4ae4-b577-dfbefd9606a4 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:50.851867 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588771fa-ef5e-4ae4-b577-dfbefd9606a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:50.852142 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588771fa-ef5e-4ae4-b577-dfbefd9606a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:50.852804 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588771fa-ef5e-4ae4-b577-dfbefd9606a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:50.853104 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588771fa-ef5e-4ae4-b577-dfbefd9606a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:50.854586 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588771fa-ef5e-4ae4-b577-dfbefd9606a4 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:50.864722 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 620/1250] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:57:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:50.871326 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80eb71-9ae0-462f-b47f-a97a7bc6c27f None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:50.955203 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:50.985630 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80eb71-9ae0-462f-b47f-a97a7bc6c27f None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:51.025592 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f0a282-58cd-4c53-acea-1b4220d09626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:51.026371 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f0a282-58cd-4c53-acea-1b4220d09626 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:51.026655 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f0a282-58cd-4c53-acea-1b4220d09626 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:51.026970 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f0a282-58cd-4c53-acea-1b4220d09626 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:51.038360 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f0a282-58cd-4c53-acea-1b4220d09626 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:51.039971 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f0a282-58cd-4c53-acea-1b4220d09626 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:51.055949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-68f0a282-58cd-4c53-acea-1b4220d09626 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:57:51.055949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:51.055949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:51.055949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:51.055949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:51.055949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:51.055949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:51.055949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:51.055949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:51.055949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:51.055949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:51.055949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:51.055949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:57:51.055949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:57:51.055949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:57:51.055949 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:57:51.057859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:51.057859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:51.057859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:51.057859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:51.057859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:51.057859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:51.057859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:51.057859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:51.057859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:51.057859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:51.057859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:51.057859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:51.057859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:51.057859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:51.057859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:51.057859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:51.057859 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:57:51.059032 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:57:51.059032 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:57:51.059032 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:57:51.059032 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 631/1251] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:57:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:57:51.065922 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588771fa-ef5e-4ae4-b577-dfbefd9606a4 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:51.174540 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588771fa-ef5e-4ae4-b577-dfbefd9606a4 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:51.211506 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb1fc5-cbd1-4dee-b741-066e26260a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:51.212126 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb1fc5-cbd1-4dee-b741-066e26260a40 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:51.212403 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb1fc5-cbd1-4dee-b741-066e26260a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:51.212677 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb1fc5-cbd1-4dee-b741-066e26260a40 None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:51.213551 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bb1fc5-cbd1-4dee-b741-066e26260a40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:51.214011 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bb1fc5-cbd1-4dee-b741-066e26260a40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:51.215431 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bb1fc5-cbd1-4dee-b741-066e26260a40 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:51.228108 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 621/1252] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:57:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:51.234669 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f0a282-58cd-4c53-acea-1b4220d09626 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:51.342440 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f0a282-58cd-4c53-acea-1b4220d09626 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:51.370655 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:51.379379 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6443d07-5317-423d-943a-4b8bdb904f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:51.380028 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6443d07-5317-423d-943a-4b8bdb904f0e None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:51.380311 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6443d07-5317-423d-943a-4b8bdb904f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:51.380584 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6443d07-5317-423d-943a-4b8bdb904f0e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:51.389737 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6443d07-5317-423d-943a-4b8bdb904f0e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:51.391268 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6443d07-5317-423d-943a-4b8bdb904f0e None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:51.404621 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-f6443d07-5317-423d-943a-4b8bdb904f0e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 14:57:51.404621 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:51.404621 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:51.404621 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:51.404621 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:51.404621 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:51.404621 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:51.404621 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:51.404621 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:51.404621 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:51.404621 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:51.404621 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:51.404621 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:57:51.404621 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:57:51.404621 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:57:51.404621 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:57:51.405866 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:51.405866 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:51.405866 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:57:51.405866 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:57:51.405866 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:57:51.405866 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:57:51.405866 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:51.405866 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:51.405866 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:51.405866 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:51.405866 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:51.405866 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:51.405866 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:51.405866 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:51.405866 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:51.405866 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:51.405866 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:51.405866 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:51.406896 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:51.406896 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:51.406896 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:57:51.406896 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:57:51.406896 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:57:51.406896 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:57:51.406896 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:57:51.406896 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:57:51.406896 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 14:57:51.406896 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:57:51.406896 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 632/1253] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:57:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:57:51.412079 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb1fc5-cbd1-4dee-b741-066e26260a40 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:51.471976 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:51.518781 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb1fc5-cbd1-4dee-b741-066e26260a40 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:51.553928 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d84a770-a499-42c1-8373-9815687a28f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:51.554535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84a770-a499-42c1-8373-9815687a28f2 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:51.554826 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84a770-a499-42c1-8373-9815687a28f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:51.555112 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84a770-a499-42c1-8373-9815687a28f2 None admin] PATH_INFO: `/v3/users` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:51.555842 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d84a770-a499-42c1-8373-9815687a28f2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:51.556145 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d84a770-a499-42c1-8373-9815687a28f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:51.558138 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d84a770-a499-42c1-8373-9815687a28f2 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:51.580113 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 622/1254] 10.222.0.33 () {56 vars in 1143 bytes} [Sun Sep 11 14:57:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:51.586474 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6443d07-5317-423d-943a-4b8bdb904f0e None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:51.695115 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6443d07-5317-423d-943a-4b8bdb904f0e None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:51.730397 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b98b7c-1c3d-49bd-8d38-914432e904a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:51.731055 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b98b7c-1c3d-49bd-8d38-914432e904a0 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:51.731339 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b98b7c-1c3d-49bd-8d38-914432e904a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:51.731610 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b98b7c-1c3d-49bd-8d38-914432e904a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:51.786954 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:51.800324 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b98b7c-1c3d-49bd-8d38-914432e904a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:51.801781 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b98b7c-1c3d-49bd-8d38-914432e904a0 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:51.896594 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 633/1255] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:57:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 14:57:51.902640 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d84a770-a499-42c1-8373-9815687a28f2 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:52.000727 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d84a770-a499-42c1-8373-9815687a28f2 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:52.013705 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:52.034621 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8dc9d-fc74-48de-8707-8fb9ea431c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:52.035275 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8dc9d-fc74-48de-8707-8fb9ea431c4f None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:52.035557 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8dc9d-fc74-48de-8707-8fb9ea431c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:52.035829 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8dc9d-fc74-48de-8707-8fb9ea431c4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:52.046117 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf8dc9d-fc74-48de-8707-8fb9ea431c4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:52.047597 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf8dc9d-fc74-48de-8707-8fb9ea431c4f None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:52.064307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-3bf8dc9d-fc74-48de-8707-8fb9ea431c4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:57:52.064307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:52.064307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:52.064307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:52.064307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:52.064307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:52.064307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:52.064307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:52.064307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:52.064307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:52.064307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:52.064307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:52.064307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:57:52.064307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:57:52.064307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:57:52.064307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:57:52.065461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:52.065461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:52.065461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:52.065461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:52.065461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:52.065461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:52.065461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:52.065461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:52.065461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:52.065461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:52.065461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:52.065461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:52.065461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:52.065461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:52.065461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:52.065461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:52.065461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:57:52.066393 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:57:52.066393 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:57:52.066393 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:57:52.066393 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 623/1256] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:57:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:57:52.071149 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b98b7c-1c3d-49bd-8d38-914432e904a0 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:52.155950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:52.168336 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b98b7c-1c3d-49bd-8d38-914432e904a0 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:52.202384 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a8cf2-bfcb-433d-aad5-4582482112f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:52.203010 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a8cf2-bfcb-433d-aad5-4582482112f3 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:52.203291 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a8cf2-bfcb-433d-aad5-4582482112f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:52.203561 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a8cf2-bfcb-433d-aad5-4582482112f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:52.204190 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a8cf2-bfcb-433d-aad5-4582482112f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:52.204487 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a8cf2-bfcb-433d-aad5-4582482112f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:52.205877 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a8cf2-bfcb-433d-aad5-4582482112f3 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:52.217286 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 634/1257] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:57:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:52.222901 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8dc9d-fc74-48de-8707-8fb9ea431c4f None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:52.319249 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8dc9d-fc74-48de-8707-8fb9ea431c4f None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:52.351907 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b20c2-369d-473b-a692-d851d4374ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:52.352511 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b20c2-369d-473b-a692-d851d4374ea1 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:52.352787 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b20c2-369d-473b-a692-d851d4374ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:52.353057 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b20c2-369d-473b-a692-d851d4374ea1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:52.360207 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4b20c2-369d-473b-a692-d851d4374ea1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:52.361672 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4b20c2-369d-473b-a692-d851d4374ea1 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:52.373415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-9f4b20c2-369d-473b-a692-d851d4374ea1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:57:52.373415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:52.373415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:52.373415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:52.373415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:52.373415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:52.373415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:52.373415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:52.373415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:52.373415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:52.373415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:52.373415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:52.373415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:57:52.373415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:57:52.373415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:57:52.374493 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:57:52.374493 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:52.374493 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:52.374493 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:52.374493 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:52.374493 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:52.374493 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:52.374493 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:52.374493 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:52.374493 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:52.374493 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:52.374493 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:52.374493 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:52.374493 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:52.374493 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:52.374493 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:52.374493 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:52.375623 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:57:52.375623 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:57:52.375623 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:57:52.375623 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:57:52.375623 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:57:52.375623 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:57:52.375623 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:57:52.375623 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:57:52.375623 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 624/1258] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:57:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:57:52.380280 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a8cf2-bfcb-433d-aad5-4582482112f3 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:52.479339 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a8cf2-bfcb-433d-aad5-4582482112f3 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:52.511144 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50685f94-5a37-4d3c-8200-9f93705648f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:52.511725 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50685f94-5a37-4d3c-8200-9f93705648f5 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:52.512000 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50685f94-5a37-4d3c-8200-9f93705648f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:52.512267 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50685f94-5a37-4d3c-8200-9f93705648f5 None admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:52.512941 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50685f94-5a37-4d3c-8200-9f93705648f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:52.513325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50685f94-5a37-4d3c-8200-9f93705648f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:52.514766 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50685f94-5a37-4d3c-8200-9f93705648f5 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:52.526703 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 635/1259] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:57:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:52.532610 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b20c2-369d-473b-a692-d851d4374ea1 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:52.558367 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:52.641416 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b20c2-369d-473b-a692-d851d4374ea1 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:52.674048 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed708b-3cec-4ac8-8ada-f955c8733a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:52.674664 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed708b-3cec-4ac8-8ada-f955c8733a80 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:52.674976 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed708b-3cec-4ac8-8ada-f955c8733a80 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:52.675248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed708b-3cec-4ac8-8ada-f955c8733a80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:52.675969 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed708b-3cec-4ac8-8ada-f955c8733a80 None admin] RBAC: Adding query filter params (role.id=99e8fe7df8e7446680a81d4eaaa1cb68, scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176, user.id=9390c5960b404bedbead2becfdc6c7af) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:52.676275 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed708b-3cec-4ac8-8ada-f955c8733a80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:52.677722 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed708b-3cec-4ac8-8ada-f955c8733a80 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:52.683071 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 625/1260] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:57:52 2022] GET /identity/v3/role_assignments?user.id=9390c5960b404bedbead2becfdc6c7af&scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176&role.id=99e8fe7df8e7446680a81d4eaaa1cb68 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:54.849450 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4cd20-e489-48f4-a2d6-bc58c42b5fb1 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:54.849985 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4cd20-e489-48f4-a2d6-bc58c42b5fb1 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:54.849985 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4cd20-e489-48f4-a2d6-bc58c42b5fb1 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:54.851363 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 636/1261] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:57:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:57:54.856592 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93739f34-adc8-4806-834a-f00a00c53128 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:54.857219 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93739f34-adc8-4806-834a-f00a00c53128 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:54.857219 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93739f34-adc8-4806-834a-f00a00c53128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:54.875109 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-93739f34-adc8-4806-834a-f00a00c53128 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:57:54.876292 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-93739f34-adc8-4806-834a-f00a00c53128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:57:54.941268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 626/1262] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:57:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:57:55.215917 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b4cd20-e489-48f4-a2d6-bc58c42b5fb1 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:55.226663 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:55.318977 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b4cd20-e489-48f4-a2d6-bc58c42b5fb1 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:55.352096 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48124407-f54e-4944-b67d-7ea56d69e5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:55.352713 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48124407-f54e-4944-b67d-7ea56d69e5ac None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:55.352990 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48124407-f54e-4944-b67d-7ea56d69e5ac None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:55.353339 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48124407-f54e-4944-b67d-7ea56d69e5ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:55.363794 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48124407-f54e-4944-b67d-7ea56d69e5ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:57:55.368591 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-48124407-f54e-4944-b67d-7ea56d69e5ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:57:55.368591 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:55.368591 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:55.368591 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:55.368591 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:55.368591 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:55.368591 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:55.368591 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:55.368591 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:55.368591 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:55.368591 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:55.368591 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:55.368591 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:57:55.368591 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:57:55.368591 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:57:55.368591 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:57:55.369800 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:57:55.369800 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:57:55.369800 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:57:55.369800 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:57:55.369800 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:55.369800 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:55.369800 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:55.369800 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:55.369800 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:55.369800 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:55.369800 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:55.369800 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:55.369800 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:55.369800 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:55.369800 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:55.369800 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:55.371690 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:55.371690 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:55.371690 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:55.371690 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:55.371690 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:57:55.371690 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:57:55.371690 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:57:55.371690 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:57:55.371690 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:57:55.371690 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:57:55.371690 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:57:55.371690 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:57:55.371690 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 637/1263] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:57:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:57:55.375973 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93739f34-adc8-4806-834a-f00a00c53128 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:55.393873 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:55.472500 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93739f34-adc8-4806-834a-f00a00c53128 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:55.505825 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780afac5-582e-46f0-90ca-166197c56bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:55.506421 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780afac5-582e-46f0-90ca-166197c56bcc None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:55.506698 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780afac5-582e-46f0-90ca-166197c56bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:55.507001 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780afac5-582e-46f0-90ca-166197c56bcc None admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:55.507656 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780afac5-582e-46f0-90ca-166197c56bcc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:55.507958 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780afac5-582e-46f0-90ca-166197c56bcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:55.509327 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780afac5-582e-46f0-90ca-166197c56bcc None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:55.517237 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 627/1264] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:57:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:55.522999 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48124407-f54e-4944-b67d-7ea56d69e5ac None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:55.593236 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:55.633378 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48124407-f54e-4944-b67d-7ea56d69e5ac None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:55.673234 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b42b8-f131-4f50-9035-ed6d333f1aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:55.673861 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b42b8-f131-4f50-9035-ed6d333f1aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:55.674141 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b42b8-f131-4f50-9035-ed6d333f1aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:55.674414 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b42b8-f131-4f50-9035-ed6d333f1aa8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:55.684358 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b42b8-f131-4f50-9035-ed6d333f1aa8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:55.686082 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b42b8-f131-4f50-9035-ed6d333f1aa8 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:55.700791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-782b42b8-f131-4f50-9035-ed6d333f1aa8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:57:55.700791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:55.700791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:55.700791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:55.700791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:55.700791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:55.700791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:55.700791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:55.700791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:55.700791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:55.700791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:55.700791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:55.700791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:57:55.700791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:57:55.700791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:57:55.700791 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:57:55.701884 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:55.701884 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:55.701884 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:55.701884 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:55.701884 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:55.701884 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:55.701884 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:55.701884 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:55.701884 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:55.701884 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:55.701884 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:55.701884 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:55.701884 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:55.701884 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:55.701884 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:55.701884 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:55.701884 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:57:55.702786 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:57:55.702786 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:57:55.702786 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:57:55.702786 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 638/1265] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:57:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:57:55.707756 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780afac5-582e-46f0-90ca-166197c56bcc None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:55.789519 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:55.808643 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780afac5-582e-46f0-90ca-166197c56bcc None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:55.842240 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c0dd2-869f-48c6-aed1-c400e77f3642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:55.842863 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c0dd2-869f-48c6-aed1-c400e77f3642 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:55.843140 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c0dd2-869f-48c6-aed1-c400e77f3642 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:55.843408 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c0dd2-869f-48c6-aed1-c400e77f3642 None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:55.844032 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702c0dd2-869f-48c6-aed1-c400e77f3642 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:55.844328 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702c0dd2-869f-48c6-aed1-c400e77f3642 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:55.845682 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702c0dd2-869f-48c6-aed1-c400e77f3642 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:55.857265 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 628/1266] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:57:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:55.863001 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b42b8-f131-4f50-9035-ed6d333f1aa8 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:55.962847 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b42b8-f131-4f50-9035-ed6d333f1aa8 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:55.982344 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:55.997622 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba3897-362a-45e3-85b4-098fda4668c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:55.998253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba3897-362a-45e3-85b4-098fda4668c3 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:55.998534 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba3897-362a-45e3-85b4-098fda4668c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:55.998820 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba3897-362a-45e3-85b4-098fda4668c3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:56.006227 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba3897-362a-45e3-85b4-098fda4668c3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:56.007726 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba3897-362a-45e3-85b4-098fda4668c3 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:56.019192 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-1aba3897-362a-45e3-85b4-098fda4668c3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 14:57:56.019192 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:56.019192 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:56.019192 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:56.019192 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:56.019192 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:56.019192 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:56.019192 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:56.019192 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:56.019192 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:56.019192 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:56.019192 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:56.019192 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:57:56.019192 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:57:56.019192 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:57:56.019192 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:57:56.020325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:56.020325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:56.020325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:57:56.020325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:57:56.020325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:57:56.020325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:57:56.020325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:56.020325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:56.020325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:56.020325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:56.020325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:56.020325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:56.020325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:56.020325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:56.020325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:56.020325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:56.020325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:56.020325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:56.021531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:56.021531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:56.021531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:57:56.021531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:57:56.021531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:57:56.021531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:57:56.021531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:57:56.021531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:57:56.021531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 14:57:56.021531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:57:56.021531 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 639/1267] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:57:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:57:56.026165 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c0dd2-869f-48c6-aed1-c400e77f3642 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:56.124152 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c0dd2-869f-48c6-aed1-c400e77f3642 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:56.157349 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce2ae88-0c25-4b0b-a4ef-08bc34f77aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:56.157945 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2ae88-0c25-4b0b-a4ef-08bc34f77aed None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:56.158225 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2ae88-0c25-4b0b-a4ef-08bc34f77aed None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:56.158496 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2ae88-0c25-4b0b-a4ef-08bc34f77aed None admin] PATH_INFO: `/v3/users` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:56.159241 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce2ae88-0c25-4b0b-a4ef-08bc34f77aed None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:56.159542 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce2ae88-0c25-4b0b-a4ef-08bc34f77aed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:56.160993 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce2ae88-0c25-4b0b-a4ef-08bc34f77aed None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:56.169649 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:56.178737 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 629/1268] 10.222.0.33 () {56 vars in 1143 bytes} [Sun Sep 11 14:57:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:56.185735 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba3897-362a-45e3-85b4-098fda4668c3 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:56.284840 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba3897-362a-45e3-85b4-098fda4668c3 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:56.317909 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38458b54-a294-41b4-af0d-6a0bc7ff6739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:56.318518 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38458b54-a294-41b4-af0d-6a0bc7ff6739 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:56.318792 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38458b54-a294-41b4-af0d-6a0bc7ff6739 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:56.319090 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38458b54-a294-41b4-af0d-6a0bc7ff6739 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:56.330018 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38458b54-a294-41b4-af0d-6a0bc7ff6739 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:56.331491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38458b54-a294-41b4-af0d-6a0bc7ff6739 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:56.345371 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-38458b54-a294-41b4-af0d-6a0bc7ff6739 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:57:56.345371 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:56.345371 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:56.345371 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:56.345371 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:56.345371 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:56.345371 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:56.345371 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:56.345371 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:56.345371 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:56.345371 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:56.345371 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:56.345371 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:57:56.345371 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:57:56.345371 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:57:56.345371 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:57:56.346445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:56.346445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:56.346445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:56.346445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:56.346445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:56.346445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:56.346445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:56.346445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:56.346445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:56.346445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:56.346445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:56.346445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:56.346445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:56.346445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:56.346445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:56.346445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:56.346445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:57:56.347580 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:57:56.347580 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:57:56.347580 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:57:56.347580 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 640/1269] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:57:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:57:56.352400 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce2ae88-0c25-4b0b-a4ef-08bc34f77aed None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:56.450018 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce2ae88-0c25-4b0b-a4ef-08bc34f77aed None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:56.482758 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f89841-93f1-484e-aeb1-e14e37f49949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:56.483381 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f89841-93f1-484e-aeb1-e14e37f49949 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:56.483657 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f89841-93f1-484e-aeb1-e14e37f49949 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:56.483926 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f89841-93f1-484e-aeb1-e14e37f49949 None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:56.484550 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f89841-93f1-484e-aeb1-e14e37f49949 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:56.484843 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f89841-93f1-484e-aeb1-e14e37f49949 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:56.486198 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f89841-93f1-484e-aeb1-e14e37f49949 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:56.497082 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 630/1270] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:57:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:56.502855 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38458b54-a294-41b4-af0d-6a0bc7ff6739 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:56.534695 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:56.607591 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38458b54-a294-41b4-af0d-6a0bc7ff6739 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:56.640560 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce4bf0-46bd-44f5-8121-682c5a8f0b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:56.641180 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce4bf0-46bd-44f5-8121-682c5a8f0b90 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:56.641459 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce4bf0-46bd-44f5-8121-682c5a8f0b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:56.641730 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce4bf0-46bd-44f5-8121-682c5a8f0b90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:56.648185 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce4bf0-46bd-44f5-8121-682c5a8f0b90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:56.649624 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce4bf0-46bd-44f5-8121-682c5a8f0b90 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:56.661318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-9dce4bf0-46bd-44f5-8121-682c5a8f0b90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:57:56.661318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:56.661318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:56.661318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:56.661318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:56.661318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:56.661318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:56.661318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:56.661318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:56.661318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:56.661318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:56.661318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:56.661318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:57:56.661318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:57:56.661318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:57:56.662459 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:57:56.662459 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:56.662459 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:56.662459 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:56.662459 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:56.662459 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:56.662459 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:56.662459 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:56.662459 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:56.662459 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:56.662459 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:56.662459 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:56.662459 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:56.662459 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:56.662459 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:56.662459 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:56.662459 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:56.663624 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:57:56.663624 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:57:56.663624 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:57:56.663624 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:57:56.663624 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:57:56.663624 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:57:56.663624 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:57:56.663624 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:57:56.663624 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 641/1271] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:57:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:57:56.669398 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f89841-93f1-484e-aeb1-e14e37f49949 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:56.727436 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:56.769307 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f89841-93f1-484e-aeb1-e14e37f49949 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:56.801405 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c25dc7-5cb7-43c1-995a-9d749ce4927b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:56.802006 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c25dc7-5cb7-43c1-995a-9d749ce4927b None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:56.802288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c25dc7-5cb7-43c1-995a-9d749ce4927b None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:56.802562 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c25dc7-5cb7-43c1-995a-9d749ce4927b None admin] PATH_INFO: `/v3/projects` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:56.803277 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c25dc7-5cb7-43c1-995a-9d749ce4927b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:56.803582 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c25dc7-5cb7-43c1-995a-9d749ce4927b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:56.805022 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c25dc7-5cb7-43c1-995a-9d749ce4927b None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:56.817455 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 631/1272] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:57:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:56.824341 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce4bf0-46bd-44f5-8121-682c5a8f0b90 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:56.913349 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:56.922650 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce4bf0-46bd-44f5-8121-682c5a8f0b90 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:56.956702 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0c8b85-c1a9-4301-b69d-ce84e5c75858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:56.957347 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c8b85-c1a9-4301-b69d-ce84e5c75858 None admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:56.957630 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c8b85-c1a9-4301-b69d-ce84e5c75858 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:56.957906 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c8b85-c1a9-4301-b69d-ce84e5c75858 None admin] PATH_INFO: `/v3/projects/9e4a3d44808e4b4092cfc4ee284c0176/users/9390c5960b404bedbead2becfdc6c7af/roles/99e8fe7df8e7446680a81d4eaaa1cb68` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:56.959808 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0c8b85-c1a9-4301-b69d-ce84e5c75858 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e4a3d44808e4b4092cfc4ee284c0176, user_id=9390c5960b404bedbead2becfdc6c7af, role_id=99e8fe7df8e7446680a81d4eaaa1cb68)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:56.961248 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0c8b85-c1a9-4301-b69d-ce84e5c75858 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:56.973410 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 642/1273] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 14:57:56 2022] PUT /identity/v3/projects/9e4a3d44808e4b4092cfc4ee284c0176/users/9390c5960b404bedbead2becfdc6c7af/roles/99e8fe7df8e7446680a81d4eaaa1cb68 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:57:59.103226 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeeddf2-fbc2-477e-9317-10c8cfa7d9cf None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:59.103776 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeeddf2-fbc2-477e-9317-10c8cfa7d9cf None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:59.103776 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeeddf2-fbc2-477e-9317-10c8cfa7d9cf None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:59.105124 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 632/1274] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:57:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:57:59.110625 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4be778-a60c-4ec3-a1c8-c714f2f26cb3 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:59.111037 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4be778-a60c-4ec3-a1c8-c714f2f26cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:59.111209 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4be778-a60c-4ec3-a1c8-c714f2f26cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:59.129470 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-0c4be778-a60c-4ec3-a1c8-c714f2f26cb3 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:57:59.130650 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-0c4be778-a60c-4ec3-a1c8-c714f2f26cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:57:59.195380 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 643/1275] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:57:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:57:59.467841 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeeddf2-fbc2-477e-9317-10c8cfa7d9cf None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:59.566704 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeeddf2-fbc2-477e-9317-10c8cfa7d9cf None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:59.584724 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:59.604603 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcce484-86f7-43f1-a28d-35f5a825f191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:59.605437 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcce484-86f7-43f1-a28d-35f5a825f191 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:59.605586 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcce484-86f7-43f1-a28d-35f5a825f191 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:59.605952 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcce484-86f7-43f1-a28d-35f5a825f191 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:59.616898 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcce484-86f7-43f1-a28d-35f5a825f191 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:57:59.622448 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-9fcce484-86f7-43f1-a28d-35f5a825f191 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:57:59.622448 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:59.622448 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:59.622448 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:59.622448 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:59.622448 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:59.622448 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:59.622448 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:59.622448 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:59.622448 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:59.622448 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:59.622448 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:59.622448 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:57:59.622448 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:57:59.622448 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:57:59.622448 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:57:59.623527 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:57:59.623527 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:57:59.623527 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:57:59.623527 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:57:59.623527 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:59.623527 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:59.623527 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:59.623527 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:59.623527 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:59.623527 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:59.623527 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:59.623527 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:59.623527 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:59.623527 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:59.623527 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:59.623527 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:59.624410 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:59.624410 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:59.624410 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:59.624410 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:59.624410 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:57:59.624410 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:57:59.624410 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:57:59.624410 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:57:59.624410 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:57:59.624410 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:57:59.624410 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:57:59.624410 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:57:59.624410 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 633/1276] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 14:57:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:57:59.630039 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4be778-a60c-4ec3-a1c8-c714f2f26cb3 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:59.715197 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:59.734284 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4be778-a60c-4ec3-a1c8-c714f2f26cb3 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:59.767130 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0091eb6-a006-4869-95e2-eadc6206ef00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:59.767726 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0091eb6-a006-4869-95e2-eadc6206ef00 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:59.768004 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0091eb6-a006-4869-95e2-eadc6206ef00 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:59.768275 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0091eb6-a006-4869-95e2-eadc6206ef00 None admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:59.768928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0091eb6-a006-4869-95e2-eadc6206ef00 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:59.769228 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0091eb6-a006-4869-95e2-eadc6206ef00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:59.770605 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0091eb6-a006-4869-95e2-eadc6206ef00 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:59.779101 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 644/1277] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 14:57:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:59.784910 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcce484-86f7-43f1-a28d-35f5a825f191 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:59.882187 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcce484-86f7-43f1-a28d-35f5a825f191 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:59.914935 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774c12c-699b-4170-9449-22d2f7d5669d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:59.915544 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774c12c-699b-4170-9449-22d2f7d5669d None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:59.916457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774c12c-699b-4170-9449-22d2f7d5669d None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:59.916794 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774c12c-699b-4170-9449-22d2f7d5669d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:59.926295 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e774c12c-699b-4170-9449-22d2f7d5669d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:59.927762 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e774c12c-699b-4170-9449-22d2f7d5669d None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:59.942422 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-e774c12c-699b-4170-9449-22d2f7d5669d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:57:59.942422 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:59.942422 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:59.942422 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:59.942422 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:59.942422 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:59.942422 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:59.942422 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:59.942422 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:59.942422 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:59.942422 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:59.942422 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:59.942422 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:57:59.942422 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:57:59.942422 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:57:59.942422 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:57:59.943601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:59.943601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:59.943601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:59.943601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:59.943601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:59.943601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:59.943601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:59.943601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:59.943601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:59.943601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:59.943601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:59.943601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:59.943601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:59.943601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:59.943601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:59.943601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:59.943601 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:57:59.944868 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:57:59.944868 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:57:59.944868 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:57:59.944868 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 634/1278] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:57:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:57:59.949432 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0091eb6-a006-4869-95e2-eadc6206ef00 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:00.049391 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0091eb6-a006-4869-95e2-eadc6206ef00 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:00.082592 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5ad0d-2a99-493c-aded-e4710aa11586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:00.083207 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5ad0d-2a99-493c-aded-e4710aa11586 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:00.083483 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5ad0d-2a99-493c-aded-e4710aa11586 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:00.083749 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5ad0d-2a99-493c-aded-e4710aa11586 None admin] PATH_INFO: `/v3/domains` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:00.084387 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b5ad0d-2a99-493c-aded-e4710aa11586 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:00.084679 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b5ad0d-2a99-493c-aded-e4710aa11586 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:00.086046 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b5ad0d-2a99-493c-aded-e4710aa11586 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:00.098551 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 645/1279] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:00.104534 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774c12c-699b-4170-9449-22d2f7d5669d None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:00.136255 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:00.203997 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774c12c-699b-4170-9449-22d2f7d5669d None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:00.236034 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350b4f6-989a-4bfb-b93d-cc6f4cccbc36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:00.236643 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350b4f6-989a-4bfb-b93d-cc6f4cccbc36 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:00.236915 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350b4f6-989a-4bfb-b93d-cc6f4cccbc36 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:00.237183 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350b4f6-989a-4bfb-b93d-cc6f4cccbc36 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:00.244305 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b350b4f6-989a-4bfb-b93d-cc6f4cccbc36 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:00.245680 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b350b4f6-989a-4bfb-b93d-cc6f4cccbc36 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:00.256390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-b350b4f6-989a-4bfb-b93d-cc6f4cccbc36 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 14:58:00.256390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:00.256390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:00.256390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:00.256390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:00.256390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:00.256390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:00.256390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:00.256390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:00.256390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:00.256390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:00.256390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:00.256390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:58:00.256390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:58:00.256390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:58:00.256390 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:58:00.257539 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:00.257539 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:00.257539 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:58:00.257539 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:58:00.257539 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:58:00.257539 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:58:00.257539 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:00.257539 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:00.257539 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:00.257539 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:00.257539 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:00.257539 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:00.257539 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:00.257539 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:00.257539 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:00.257539 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:00.257539 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:00.257539 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:00.258683 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:00.258683 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:00.258683 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:58:00.258683 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:58:00.258683 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:58:00.258683 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:58:00.258683 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:58:00.258683 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:58:00.258683 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 14:58:00.258683 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:58:00.258683 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 635/1280] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:58:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:00.263797 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5ad0d-2a99-493c-aded-e4710aa11586 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:00.267589 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:00.362635 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5ad0d-2a99-493c-aded-e4710aa11586 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:00.395748 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e01b1-0bc6-40c1-b75b-931b8dca0e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:00.396344 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e01b1-0bc6-40c1-b75b-931b8dca0e0f None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:00.396626 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e01b1-0bc6-40c1-b75b-931b8dca0e0f None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:00.396901 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e01b1-0bc6-40c1-b75b-931b8dca0e0f None admin] PATH_INFO: `/v3/users` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:00.397621 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430e01b1-0bc6-40c1-b75b-931b8dca0e0f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:00.397926 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430e01b1-0bc6-40c1-b75b-931b8dca0e0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:00.399396 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430e01b1-0bc6-40c1-b75b-931b8dca0e0f None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:00.416851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 646/1281] 10.222.0.33 () {56 vars in 1143 bytes} [Sun Sep 11 14:58:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:00.422303 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350b4f6-989a-4bfb-b93d-cc6f4cccbc36 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:00.511820 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:00.521733 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350b4f6-989a-4bfb-b93d-cc6f4cccbc36 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:00.555268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0269e3-b2f8-4a37-ac42-c62febfcc14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:00.555861 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0269e3-b2f8-4a37-ac42-c62febfcc14a None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:00.556147 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0269e3-b2f8-4a37-ac42-c62febfcc14a None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:00.556467 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0269e3-b2f8-4a37-ac42-c62febfcc14a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:00.627909 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0269e3-b2f8-4a37-ac42-c62febfcc14a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:00.629349 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0269e3-b2f8-4a37-ac42-c62febfcc14a None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:00.714816 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:00.725107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 636/1282] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 14:58:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 14:58:00.731775 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e01b1-0bc6-40c1-b75b-931b8dca0e0f None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:00.791817 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:00.829902 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e01b1-0bc6-40c1-b75b-931b8dca0e0f None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:00.863153 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c29f7-b230-446f-b25e-369d7fc50e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:00.863775 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c29f7-b230-446f-b25e-369d7fc50e85 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:00.864054 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c29f7-b230-446f-b25e-369d7fc50e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:00.864327 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c29f7-b230-446f-b25e-369d7fc50e85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:00.874004 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c29f7-b230-446f-b25e-369d7fc50e85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:00.875486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c29f7-b230-446f-b25e-369d7fc50e85 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:00.890250 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-ff9c29f7-b230-446f-b25e-369d7fc50e85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:58:00.890250 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:00.890250 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:00.890250 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:00.890250 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:00.890250 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:00.890250 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:00.890250 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:00.890250 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:00.890250 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:00.890250 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:00.890250 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:00.890250 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:58:00.890250 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:58:00.890250 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:58:00.890250 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:58:00.891325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:00.891325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:00.891325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:00.891325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:00.891325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:00.891325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:00.891325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:00.891325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:00.891325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:00.891325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:00.891325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:00.891325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:00.891325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:00.891325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:00.891325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:00.891325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:00.891325 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:58:00.892256 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:58:00.892256 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:58:00.892256 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:58:00.892256 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 647/1283] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 14:58:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:00.897113 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0269e3-b2f8-4a37-ac42-c62febfcc14a None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:00.994794 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0269e3-b2f8-4a37-ac42-c62febfcc14a None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:01.027810 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67776027-45f2-4793-84a6-1598ee8a2238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:01.028404 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67776027-45f2-4793-84a6-1598ee8a2238 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:01.028684 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67776027-45f2-4793-84a6-1598ee8a2238 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:01.028956 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67776027-45f2-4793-84a6-1598ee8a2238 None admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:01.029587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67776027-45f2-4793-84a6-1598ee8a2238 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:01.029888 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67776027-45f2-4793-84a6-1598ee8a2238 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:01.031293 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67776027-45f2-4793-84a6-1598ee8a2238 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:01.042663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 637/1284] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 14:58:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:01.048202 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c29f7-b230-446f-b25e-369d7fc50e85 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:01.146033 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c29f7-b230-446f-b25e-369d7fc50e85 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:01.159349 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:01.179473 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da3603-b904-43a9-a67b-abc46f1c8a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:01.180080 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da3603-b904-43a9-a67b-abc46f1c8a0b None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:01.180357 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da3603-b904-43a9-a67b-abc46f1c8a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:01.180625 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da3603-b904-43a9-a67b-abc46f1c8a0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:01.188116 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da3603-b904-43a9-a67b-abc46f1c8a0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:01.189564 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da3603-b904-43a9-a67b-abc46f1c8a0b None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:01.200761 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application [None req-f7da3603-b904-43a9-a67b-abc46f1c8a0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:58:01.200761 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:01.200761 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:01.200761 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:01.200761 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:01.200761 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:01.200761 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:01.200761 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:01.200761 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:01.200761 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:01.200761 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:01.200761 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:01.200761 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:58:01.200761 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:58:01.200761 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:58:01.201788 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:58:01.201788 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:01.201788 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:01.201788 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:01.201788 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:01.201788 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:01.201788 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:01.201788 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:01.201788 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:01.201788 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:01.201788 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:01.201788 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:01.201788 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:01.201788 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:01.201788 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:01.201788 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:01.201788 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:01.203073 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:58:01.203073 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:58:01.203073 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:58:01.203073 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:58:01.203073 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:58:01.203073 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:58:01.203073 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:58:01.203073 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: ERROR keystone.server.flask.application Sep 11 14:58:01.203073 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 648/1285] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 14:58:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:01.207577 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67776027-45f2-4793-84a6-1598ee8a2238 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:01.257539 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:01.307610 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67776027-45f2-4793-84a6-1598ee8a2238 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:01.340449 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75363e94-2044-4ef2-9477-5b5a4fd7ad0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:01.341042 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75363e94-2044-4ef2-9477-5b5a4fd7ad0b None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:01.341319 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75363e94-2044-4ef2-9477-5b5a4fd7ad0b None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:01.341588 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75363e94-2044-4ef2-9477-5b5a4fd7ad0b None admin] PATH_INFO: `/v3/projects` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:01.342264 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75363e94-2044-4ef2-9477-5b5a4fd7ad0b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:01.342570 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75363e94-2044-4ef2-9477-5b5a4fd7ad0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:01.344045 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75363e94-2044-4ef2-9477-5b5a4fd7ad0b None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:01.356790 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 638/1286] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 14:58:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:01.362743 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da3603-b904-43a9-a67b-abc46f1c8a0b None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:01.461042 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da3603-b904-43a9-a67b-abc46f1c8a0b None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:01.494794 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60213e8-29ae-4b0b-b118-fc78b2e8ea90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:01.495402 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60213e8-29ae-4b0b-b118-fc78b2e8ea90 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:01.495676 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60213e8-29ae-4b0b-b118-fc78b2e8ea90 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:01.495941 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60213e8-29ae-4b0b-b118-fc78b2e8ea90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:01.496645 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60213e8-29ae-4b0b-b118-fc78b2e8ea90 None admin] RBAC: Adding query filter params (role.id=99e8fe7df8e7446680a81d4eaaa1cb68, scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176, user.id=9390c5960b404bedbead2becfdc6c7af) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:01.496945 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60213e8-29ae-4b0b-b118-fc78b2e8ea90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:01.498367 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60213e8-29ae-4b0b-b118-fc78b2e8ea90 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:01.503980 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 649/1287] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 14:58:01 2022] GET /identity/v3/role_assignments?user.id=9390c5960b404bedbead2becfdc6c7af&scope.project.id=9e4a3d44808e4b4092cfc4ee284c0176&role.id=99e8fe7df8e7446680a81d4eaaa1cb68 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:03.670343 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f3e3a-dda5-4f08-9807-92b0c7d6285a None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:03.670951 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f3e3a-dda5-4f08-9807-92b0c7d6285a None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:03.670951 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f3e3a-dda5-4f08-9807-92b0c7d6285a None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:03.672231 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 639/1288] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:03.677307 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b8aa5-bc9f-4829-9691-378444f1a6fe None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:03.677687 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b8aa5-bc9f-4829-9691-378444f1a6fe None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:03.677850 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b8aa5-bc9f-4829-9691-378444f1a6fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:03.696874 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-9b0b8aa5-bc9f-4829-9691-378444f1a6fe None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:03.698694 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-9b0b8aa5-bc9f-4829-9691-378444f1a6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:03.727379 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:03.765401 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 650/1289] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:58:04.030230 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f3e3a-dda5-4f08-9807-92b0c7d6285a None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:04.131937 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f3e3a-dda5-4f08-9807-92b0c7d6285a None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:04.142182 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:04.165249 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd1a152-4b15-41fe-b87e-87088beb44aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:04.166610 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1a152-4b15-41fe-b87e-87088beb44aa None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:04.167075 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1a152-4b15-41fe-b87e-87088beb44aa None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:04.167352 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1a152-4b15-41fe-b87e-87088beb44aa None admin] PATH_INFO: `/v3/services/placement` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:04.167924 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd1a152-4b15-41fe-b87e-87088beb44aa None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:04.169281 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd1a152-4b15-41fe-b87e-87088beb44aa None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:04.177804 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-1bd1a152-4b15-41fe-b87e-87088beb44aa None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 14:58:04.177804 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:04.177804 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:58:04.177804 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:58:04.177804 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:58:04.177804 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:58:04.177804 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:58:04.177804 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:58:04.177804 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 14:58:04.177804 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:58:04.177804 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:58:04.177804 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:58:04.177804 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:04.177804 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:04.177804 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:04.177804 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:04.177804 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:04.179088 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:04.179088 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:04.179088 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:04.179088 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:04.179088 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:04.179088 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:04.179088 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:58:04.179088 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:58:04.179088 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:58:04.179088 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:58:04.179088 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:04.179088 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:04.179088 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:04.179088 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:04.179088 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:04.179088 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:04.180248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:04.180248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:04.180248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:04.180248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:04.180248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:04.180248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:04.180248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:04.180248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:04.180248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:04.180248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:04.180248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:58:04.180248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:58:04.180248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 14:58:04.180248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:58:04.180248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 640/1290] 10.222.0.33 () {56 vars in 1117 bytes} [Sun Sep 11 14:58:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:04.185858 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b8aa5-bc9f-4829-9691-378444f1a6fe None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:04.281294 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b8aa5-bc9f-4829-9691-378444f1a6fe None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:04.314596 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894d1f8-46aa-4ca4-9dda-c1d432507f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:04.315232 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894d1f8-46aa-4ca4-9dda-c1d432507f5d None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:04.315509 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894d1f8-46aa-4ca4-9dda-c1d432507f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:04.315776 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894d1f8-46aa-4ca4-9dda-c1d432507f5d None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:04.316395 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e894d1f8-46aa-4ca4-9dda-c1d432507f5d None admin] RBAC: Adding query filter params (name=placement) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:04.316689 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e894d1f8-46aa-4ca4-9dda-c1d432507f5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:04.318075 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e894d1f8-46aa-4ca4-9dda-c1d432507f5d None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:04.323247 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 651/1291] 10.222.0.33 () {56 vars in 1116 bytes} [Sun Sep 11 14:58:04 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:04.328907 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd1a152-4b15-41fe-b87e-87088beb44aa None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:04.424786 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd1a152-4b15-41fe-b87e-87088beb44aa None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:04.457230 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744c24ee-2747-4977-b25f-c97599d751f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:04.457812 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c24ee-2747-4977-b25f-c97599d751f6 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:04.458085 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c24ee-2747-4977-b25f-c97599d751f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:04.458352 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c24ee-2747-4977-b25f-c97599d751f6 None admin] PATH_INFO: `/v3/services` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:04.458997 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744c24ee-2747-4977-b25f-c97599d751f6 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:04.459294 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744c24ee-2747-4977-b25f-c97599d751f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:04.460640 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744c24ee-2747-4977-b25f-c97599d751f6 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:04.465415 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 641/1292] 10.222.0.33 () {56 vars in 1116 bytes} [Sun Sep 11 14:58:04 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:06.600223 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b8203-6f4b-4d07-ab9d-e717f43b6a34 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:06.600766 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b8203-6f4b-4d07-ab9d-e717f43b6a34 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:06.600766 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b8203-6f4b-4d07-ab9d-e717f43b6a34 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:06.602107 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 652/1293] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:58:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:06.607285 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce36354-3a94-4f88-a6e9-892fad488cb4 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:06.607718 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce36354-3a94-4f88-a6e9-892fad488cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:06.607812 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce36354-3a94-4f88-a6e9-892fad488cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:06.626465 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-9ce36354-3a94-4f88-a6e9-892fad488cb4 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:06.627693 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-9ce36354-3a94-4f88-a6e9-892fad488cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:06.656737 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:06.693855 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 642/1294] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:58:06.958314 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b8203-6f4b-4d07-ab9d-e717f43b6a34 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:06.989480 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:07.059926 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b8203-6f4b-4d07-ab9d-e717f43b6a34 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:07.094384 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bd67bd-33d2-4bea-80fd-56e144162e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:07.095004 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd67bd-33d2-4bea-80fd-56e144162e32 None admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:07.095281 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd67bd-33d2-4bea-80fd-56e144162e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:07.095550 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd67bd-33d2-4bea-80fd-56e144162e32 None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:07.096197 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bd67bd-33d2-4bea-80fd-56e144162e32 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:07.097557 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bd67bd-33d2-4bea-80fd-56e144162e32 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:07.111983 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 653/1295] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 11 14:58:06 2022] POST /identity/v3/services => generated 246 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:58:09.301436 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07480f-4b2a-4d90-887d-bef457bfa41d None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:09.301947 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07480f-4b2a-4d90-887d-bef457bfa41d None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:09.301947 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07480f-4b2a-4d90-887d-bef457bfa41d None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:09.303270 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 643/1296] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:09.308971 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78a79c-1fc5-4ade-a87e-d6de40dfd622 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:09.309370 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78a79c-1fc5-4ade-a87e-d6de40dfd622 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:09.309519 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78a79c-1fc5-4ade-a87e-d6de40dfd622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:09.327819 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-ca78a79c-1fc5-4ade-a87e-d6de40dfd622 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:09.329025 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-ca78a79c-1fc5-4ade-a87e-d6de40dfd622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:09.403565 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 654/1297] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:09 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:58:09.674573 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07480f-4b2a-4d90-887d-bef457bfa41d None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:09.774417 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07480f-4b2a-4d90-887d-bef457bfa41d None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:09.808532 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2044631-4717-4077-93fc-e71fb7741021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:09.809149 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2044631-4717-4077-93fc-e71fb7741021 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:09.809427 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2044631-4717-4077-93fc-e71fb7741021 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:09.809698 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2044631-4717-4077-93fc-e71fb7741021 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:09.810269 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2044631-4717-4077-93fc-e71fb7741021 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:09.811919 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2044631-4717-4077-93fc-e71fb7741021 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:09.820498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-a2044631-4717-4077-93fc-e71fb7741021 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 14:58:09.820498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:09.820498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:58:09.820498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:58:09.820498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:58:09.820498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:58:09.820498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:58:09.820498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:58:09.820498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 14:58:09.820498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:58:09.820498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:58:09.820498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:58:09.820498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:09.820498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:09.820498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:09.820498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:09.820498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:09.822217 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:09.822217 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:09.822217 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:09.822217 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:09.822217 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:09.822217 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:09.822217 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:58:09.822217 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:58:09.822217 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:58:09.822217 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:58:09.822217 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:09.822217 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:09.822217 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:09.822217 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:09.822217 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:09.822217 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:09.823195 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:09.823195 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:09.823195 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:09.823195 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:09.823195 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:09.823195 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:09.823195 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:09.823195 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:09.823195 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:09.823195 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:09.823195 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:58:09.823195 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:58:09.823195 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 14:58:09.823195 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:58:09.823195 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 644/1298] 10.222.0.33 () {56 vars in 1117 bytes} [Sun Sep 11 14:58:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:09.828034 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca78a79c-1fc5-4ade-a87e-d6de40dfd622 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:09.846438 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:09.927816 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca78a79c-1fc5-4ade-a87e-d6de40dfd622 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:09.961258 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df466614-9c49-400b-aaaf-c071bb7915b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:09.961855 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df466614-9c49-400b-aaaf-c071bb7915b3 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:09.962136 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df466614-9c49-400b-aaaf-c071bb7915b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:09.963078 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df466614-9c49-400b-aaaf-c071bb7915b3 None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:09.963732 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df466614-9c49-400b-aaaf-c071bb7915b3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:09.964034 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df466614-9c49-400b-aaaf-c071bb7915b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:09.965412 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df466614-9c49-400b-aaaf-c071bb7915b3 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:09.970384 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 655/1299] 10.222.0.33 () {56 vars in 1116 bytes} [Sun Sep 11 14:58:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:09.976380 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2044631-4717-4077-93fc-e71fb7741021 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:10.008948 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:10.076106 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2044631-4717-4077-93fc-e71fb7741021 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:10.108629 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21443909-7bb7-4f45-8910-7ede84cca3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:10.109220 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21443909-7bb7-4f45-8910-7ede84cca3d7 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:10.109496 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21443909-7bb7-4f45-8910-7ede84cca3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:10.109769 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21443909-7bb7-4f45-8910-7ede84cca3d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:10.110472 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21443909-7bb7-4f45-8910-7ede84cca3d7 None admin] RBAC: Adding query filter params (interface=public, service_id=210470445c884bc093a2588c8338845b, region_id=RegionOne) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:10.110775 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21443909-7bb7-4f45-8910-7ede84cca3d7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:10.112529 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21443909-7bb7-4f45-8910-7ede84cca3d7 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:10.117463 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 645/1300] 10.222.0.33 () {56 vars in 1251 bytes} [Sun Sep 11 14:58:09 2022] GET /identity/v3/endpoints?service_id=210470445c884bc093a2588c8338845b&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:10.124056 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df466614-9c49-400b-aaaf-c071bb7915b3 None admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:10.203673 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:10.222592 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df466614-9c49-400b-aaaf-c071bb7915b3 None admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:10.255909 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eab183-57fa-4b32-bce2-5debfbca7754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:10.256500 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eab183-57fa-4b32-bce2-5debfbca7754 None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:10.256779 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eab183-57fa-4b32-bce2-5debfbca7754 None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:10.257048 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eab183-57fa-4b32-bce2-5debfbca7754 None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:10.257648 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eab183-57fa-4b32-bce2-5debfbca7754 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:10.259044 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eab183-57fa-4b32-bce2-5debfbca7754 None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:10.265760 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 656/1301] 10.222.0.33 () {56 vars in 1087 bytes} [Sun Sep 11 14:58:10 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 14:58:12.518053 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492b2d8-5bc1-4517-8b23-0a16133deb98 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:12.518579 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492b2d8-5bc1-4517-8b23-0a16133deb98 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:12.518579 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492b2d8-5bc1-4517-8b23-0a16133deb98 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:12.520002 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 646/1302] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:58:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:12.525743 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95902cb-c17b-461d-9d31-826b6e8a2ae0 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:12.526143 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95902cb-c17b-461d-9d31-826b6e8a2ae0 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:12.526278 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95902cb-c17b-461d-9d31-826b6e8a2ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:12.544766 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-c95902cb-c17b-461d-9d31-826b6e8a2ae0 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:12.545938 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-c95902cb-c17b-461d-9d31-826b6e8a2ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:12.612352 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 657/1303] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:12 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:58:12.880325 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492b2d8-5bc1-4517-8b23-0a16133deb98 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:12.974937 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:12.982786 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492b2d8-5bc1-4517-8b23-0a16133deb98 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:13.017343 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e00548-83f0-4895-872d-f8ab76180307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:13.017973 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e00548-83f0-4895-872d-f8ab76180307 None admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:13.018255 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e00548-83f0-4895-872d-f8ab76180307 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:13.018530 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e00548-83f0-4895-872d-f8ab76180307 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:13.019143 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e00548-83f0-4895-872d-f8ab76180307 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:13.020515 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e00548-83f0-4895-872d-f8ab76180307 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:13.029775 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application [None req-30e00548-83f0-4895-872d-f8ab76180307 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 14:58:13.029775 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:13.029775 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:58:13.029775 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:58:13.029775 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:58:13.029775 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:58:13.029775 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:58:13.029775 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:58:13.029775 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 14:58:13.029775 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:58:13.029775 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:58:13.029775 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:58:13.029775 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:13.029775 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:13.029775 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:13.029775 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:13.029775 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:13.031149 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:13.031149 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:13.031149 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:13.031149 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:13.031149 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:13.031149 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:13.031149 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:58:13.031149 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:58:13.031149 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:58:13.031149 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:58:13.031149 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:13.031149 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:13.031149 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:13.031149 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:13.031149 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:13.031149 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:13.032262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:13.032262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:13.032262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:13.032262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:13.032262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:13.032262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:13.032262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:13.032262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:13.032262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:13.032262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:13.032262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:58:13.032262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:58:13.032262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 14:58:13.032262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: ERROR keystone.server.flask.application Sep 11 14:58:13.032262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 647/1304] 10.222.0.33 () {56 vars in 1117 bytes} [Sun Sep 11 14:58:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:13.036960 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95902cb-c17b-461d-9d31-826b6e8a2ae0 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:13.111147 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:13.134969 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95902cb-c17b-461d-9d31-826b6e8a2ae0 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:13.168429 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85324c5-4bb3-4e81-9fd8-4b22be65b16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:13.169026 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85324c5-4bb3-4e81-9fd8-4b22be65b16f None admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:13.169306 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85324c5-4bb3-4e81-9fd8-4b22be65b16f None admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:13.169584 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85324c5-4bb3-4e81-9fd8-4b22be65b16f None admin] PATH_INFO: `/v3/services` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:13.170210 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85324c5-4bb3-4e81-9fd8-4b22be65b16f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81430) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:13.170514 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85324c5-4bb3-4e81-9fd8-4b22be65b16f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:13.171928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85324c5-4bb3-4e81-9fd8-4b22be65b16f None admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:13.176934 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 658/1305] 10.222.0.33 () {56 vars in 1116 bytes} [Sun Sep 11 14:58:13 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:13.183256 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e00548-83f0-4895-872d-f8ab76180307 None admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:13.281004 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e00548-83f0-4895-872d-f8ab76180307 None admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:13.314525 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a2b6b-4e68-4685-8c74-e718100a6f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:13.315144 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a2b6b-4e68-4685-8c74-e718100a6f52 None admin] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:13.315426 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a2b6b-4e68-4685-8c74-e718100a6f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:13.315702 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a2b6b-4e68-4685-8c74-e718100a6f52 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:13.316371 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a2b6b-4e68-4685-8c74-e718100a6f52 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:13.317738 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a2b6b-4e68-4685-8c74-e718100a6f52 None admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:13.336635 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 648/1306] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 14:58:13 2022] POST /identity/v3/endpoints => generated 331 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:58:23.174048 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=30 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:23.332468 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:17.590837 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392ceff-4572-42c7-9eaf-98c50e8975c5 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:17.590837 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392ceff-4572-42c7-9eaf-98c50e8975c5 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:17.590837 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392ceff-4572-42c7-9eaf-98c50e8975c5 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:17.597775 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 659/1307] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:59:17 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:17.598959 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecfc1e-83f7-46ea-99f2-ef50ffd95d2d None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:17.599518 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecfc1e-83f7-46ea-99f2-ef50ffd95d2d None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:17.599719 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecfc1e-83f7-46ea-99f2-ef50ffd95d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:17.736861 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-75ecfc1e-83f7-46ea-99f2-ef50ffd95d2d None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:17.738017 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-75ecfc1e-83f7-46ea-99f2-ef50ffd95d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:17.873644 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 649/1308] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 14:59:18.168512 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffe816-0d2a-411c-b0c3-3e57dae6d65b None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:18.168828 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffe816-0d2a-411c-b0c3-3e57dae6d65b None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:18.169107 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffe816-0d2a-411c-b0c3-3e57dae6d65b None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:18.170527 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 660/1309] 10.222.0.33 () {54 vars in 965 bytes} [Sun Sep 11 14:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:18.177277 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11981bc-8e00-4913-a449-b0d251cc5aa4 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:18.177559 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11981bc-8e00-4913-a449-b0d251cc5aa4 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:18.177836 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11981bc-8e00-4913-a449-b0d251cc5aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:18.269430 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-c11981bc-8e00-4913-a449-b0d251cc5aa4 None None] MFA Rules not processed for user `df6eb236e80b4395a7aa77020b2f3a4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:18.270641 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-c11981bc-8e00-4913-a449-b0d251cc5aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:18.292627 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:18.356651 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 650/1310] 10.222.0.33 () {58 vars in 1062 bytes} [Sun Sep 11 14:59:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 14:59:18.365492 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bffe816-0d2a-411c-b0c3-3e57dae6d65b None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:18.367313 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-9bffe816-0d2a-411c-b0c3-3e57dae6d65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:18.483714 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bffe816-0d2a-411c-b0c3-3e57dae6d65b None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:18.563110 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c05731-3bb1-4d53-a077-0f136a687a47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6eb236e80b4395a7aa77020b2f3a4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:18.563731 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c05731-3bb1-4d53-a077-0f136a687a47 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:18.564015 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c05731-3bb1-4d53-a077-0f136a687a47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:18.564287 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c05731-3bb1-4d53-a077-0f136a687a47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:18.565943 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-b0c05731-3bb1-4d53-a077-0f136a687a47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:18.628100 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:18.691175 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c05731-3bb1-4d53-a077-0f136a687a47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:18.692648 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c05731-3bb1-4d53-a077-0f136a687a47 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:18.714052 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 661/1311] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 14:59:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 14:59:21.219325 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ecf98e-987c-41c7-b4a6-ea9008e6cc4e None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:21.219758 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ecf98e-987c-41c7-b4a6-ea9008e6cc4e None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:21.219869 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ecf98e-987c-41c7-b4a6-ea9008e6cc4e None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:21.221364 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 651/1312] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:21.226730 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057547d7-04d2-4534-a414-b4108a651f40 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:21.227259 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057547d7-04d2-4534-a414-b4108a651f40 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:21.227371 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057547d7-04d2-4534-a414-b4108a651f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:21.300171 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-057547d7-04d2-4534-a414-b4108a651f40 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:21.301401 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-057547d7-04d2-4534-a414-b4108a651f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:21.400230 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 662/1313] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 14:59:21.684777 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ecf98e-987c-41c7-b4a6-ea9008e6cc4e None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:21.686305 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-10ecf98e-987c-41c7-b4a6-ea9008e6cc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:21.794625 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ecf98e-987c-41c7-b4a6-ea9008e6cc4e None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:21.872632 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:21.875662 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f450c5f-8766-4648-bd9e-3dd117b164af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6eb236e80b4395a7aa77020b2f3a4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:21.876518 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f450c5f-8766-4648-bd9e-3dd117b164af service neutron] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:21.876978 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f450c5f-8766-4648-bd9e-3dd117b164af service neutron] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:21.877463 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f450c5f-8766-4648-bd9e-3dd117b164af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:21.879209 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-6f450c5f-8766-4648-bd9e-3dd117b164af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:22.003989 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f450c5f-8766-4648-bd9e-3dd117b164af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:22.005471 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f450c5f-8766-4648-bd9e-3dd117b164af service neutron] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:22.026795 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 652/1314] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 14:59:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 14:59:24.518398 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6d3ea-5ceb-454e-8464-ed1e6bb7ab99 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:24.518974 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6d3ea-5ceb-454e-8464-ed1e6bb7ab99 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:24.518974 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6d3ea-5ceb-454e-8464-ed1e6bb7ab99 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:24.520322 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 663/1315] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:59:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:24.526140 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c410f-01bb-4c16-badb-14ed9566d610 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:24.526569 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c410f-01bb-4c16-badb-14ed9566d610 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:24.526699 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c410f-01bb-4c16-badb-14ed9566d610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:24.594571 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-3f1c410f-01bb-4c16-badb-14ed9566d610 None None] MFA Rules not processed for user `4585768734fc4332a61c845383c1482c`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:24.595852 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-3f1c410f-01bb-4c16-badb-14ed9566d610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:24.647808 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:24.693170 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 653/1316] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:24 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 14:59:24.980932 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff6d3ea-5ceb-454e-8464-ed1e6bb7ab99 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:24.982516 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-4ff6d3ea-5ceb-454e-8464-ed1e6bb7ab99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:24.992919 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:25.086989 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff6d3ea-5ceb-454e-8464-ed1e6bb7ab99 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:25.159434 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df3442-7be7-4a7f-a822-056e70c068f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6eb236e80b4395a7aa77020b2f3a4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:25.160088 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df3442-7be7-4a7f-a822-056e70c068f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:25.160318 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df3442-7be7-4a7f-a822-056e70c068f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:25.160593 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df3442-7be7-4a7f-a822-056e70c068f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:25.162187 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-60df3442-7be7-4a7f-a822-056e70c068f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:25.226155 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:25.273673 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df3442-7be7-4a7f-a822-056e70c068f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:25.275179 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df3442-7be7-4a7f-a822-056e70c068f6 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:25.293477 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 664/1317] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 14:59:24 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 14:59:27.994110 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300415e2-3a95-4208-bedc-d478b4cb442a None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:27.994656 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300415e2-3a95-4208-bedc-d478b4cb442a None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:27.994656 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300415e2-3a95-4208-bedc-d478b4cb442a None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:27.996037 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 654/1318] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:59:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:59:28.001343 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe3df1-35ed-4bca-aa56-f999701139cb None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:28.001801 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe3df1-35ed-4bca-aa56-f999701139cb None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:28.001929 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe3df1-35ed-4bca-aa56-f999701139cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:28.066723 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-2ffe3df1-35ed-4bca-aa56-f999701139cb None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:28.067974 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-2ffe3df1-35ed-4bca-aa56-f999701139cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:28.162529 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 665/1319] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 14:59:28.450448 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300415e2-3a95-4208-bedc-d478b4cb442a None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:28.452040 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-300415e2-3a95-4208-bedc-d478b4cb442a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:28.555616 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300415e2-3a95-4208-bedc-d478b4cb442a None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:28.629582 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabcd94c-7f93-4c5c-9157-820c2b952404 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6eb236e80b4395a7aa77020b2f3a4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:28.630190 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabcd94c-7f93-4c5c-9157-820c2b952404 service neutron] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:28.630466 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabcd94c-7f93-4c5c-9157-820c2b952404 service neutron] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:28.630737 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabcd94c-7f93-4c5c-9157-820c2b952404 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:28.633462 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-fabcd94c-7f93-4c5c-9157-820c2b952404 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:28.640523 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:28.742742 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabcd94c-7f93-4c5c-9157-820c2b952404 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:28.744213 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabcd94c-7f93-4c5c-9157-820c2b952404 service neutron] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:28.762411 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 655/1320] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 14:59:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 14:59:31.556808 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d1c35-f9ca-4094-95e4-3a9ca9746340 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:31.557340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d1c35-f9ca-4094-95e4-3a9ca9746340 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:31.557340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d1c35-f9ca-4094-95e4-3a9ca9746340 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:31.558663 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 666/1321] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:59:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:31.565581 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef3d05-fb35-4699-a67f-a03fffcfe7ba None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:31.566054 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef3d05-fb35-4699-a67f-a03fffcfe7ba None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:31.566160 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef3d05-fb35-4699-a67f-a03fffcfe7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:31.642995 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-8aef3d05-fb35-4699-a67f-a03fffcfe7ba None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:31.644226 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-8aef3d05-fb35-4699-a67f-a03fffcfe7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:31.706413 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:31.748547 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 656/1322] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 14:59:32.033622 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23d1c35-f9ca-4094-95e4-3a9ca9746340 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:32.035268 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-b23d1c35-f9ca-4094-95e4-3a9ca9746340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:32.055887 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:32.140375 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23d1c35-f9ca-4094-95e4-3a9ca9746340 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:32.213451 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd1d87-5678-4dfc-881c-f22cf785238a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6eb236e80b4395a7aa77020b2f3a4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:32.214056 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd1d87-5678-4dfc-881c-f22cf785238a service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:32.214335 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd1d87-5678-4dfc-881c-f22cf785238a service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:32.214604 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd1d87-5678-4dfc-881c-f22cf785238a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:32.216219 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-56cd1d87-5678-4dfc-881c-f22cf785238a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:32.288771 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:32.327355 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd1d87-5678-4dfc-881c-f22cf785238a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:32.328808 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd1d87-5678-4dfc-881c-f22cf785238a service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:32.348095 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 667/1323] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 14:59:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 14:59:34.544248 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed250ffc-80ab-42c2-9b8b-ddd8ee46b459 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:34.544788 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed250ffc-80ab-42c2-9b8b-ddd8ee46b459 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:34.544788 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed250ffc-80ab-42c2-9b8b-ddd8ee46b459 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:34.546115 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 657/1324] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:59:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:34.551465 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32bce00-3051-4427-baf2-c76d03d275c3 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:34.551924 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32bce00-3051-4427-baf2-c76d03d275c3 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:34.552020 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32bce00-3051-4427-baf2-c76d03d275c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:34.621370 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-c32bce00-3051-4427-baf2-c76d03d275c3 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:34.622633 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-c32bce00-3051-4427-baf2-c76d03d275c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:34.733590 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 668/1325] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 14:59:35.016222 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed250ffc-80ab-42c2-9b8b-ddd8ee46b459 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:35.017806 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-ed250ffc-80ab-42c2-9b8b-ddd8ee46b459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:35.120814 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed250ffc-80ab-42c2-9b8b-ddd8ee46b459 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:35.194767 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c03ae-d818-4d18-b611-3462dfcabe53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6eb236e80b4395a7aa77020b2f3a4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:35.195429 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c03ae-d818-4d18-b611-3462dfcabe53 service neutron] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:35.195784 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c03ae-d818-4d18-b611-3462dfcabe53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:35.196036 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c03ae-d818-4d18-b611-3462dfcabe53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:35.197631 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-af2c03ae-d818-4d18-b611-3462dfcabe53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:35.214882 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:35.307086 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c03ae-d818-4d18-b611-3462dfcabe53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:35.308448 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c03ae-d818-4d18-b611-3462dfcabe53 service neutron] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:35.327407 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 658/1326] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 14:59:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 14:59:37.550134 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95b6e1-4d59-4adc-bee5-1e3b30dcdadc None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:37.550633 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95b6e1-4d59-4adc-bee5-1e3b30dcdadc None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:37.550633 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95b6e1-4d59-4adc-bee5-1e3b30dcdadc None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:37.552108 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 669/1327] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:59:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:37.557588 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7effa-93a8-40df-b3fd-86885ca77705 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:37.558014 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7effa-93a8-40df-b3fd-86885ca77705 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:37.558136 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7effa-93a8-40df-b3fd-86885ca77705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:37.625838 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-20c7effa-93a8-40df-b3fd-86885ca77705 None None] MFA Rules not processed for user `4585768734fc4332a61c845383c1482c`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:37.627134 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-20c7effa-93a8-40df-b3fd-86885ca77705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:37.692706 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:37.725075 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 659/1328] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:37 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 14:59:38.017556 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e95b6e1-4d59-4adc-bee5-1e3b30dcdadc None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:38.019120 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-0e95b6e1-4d59-4adc-bee5-1e3b30dcdadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:38.052835 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:38.127542 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e95b6e1-4d59-4adc-bee5-1e3b30dcdadc None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:38.201203 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7fb1f-852a-4df0-a12b-5e53c2718301 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6eb236e80b4395a7aa77020b2f3a4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:38.201728 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7fb1f-852a-4df0-a12b-5e53c2718301 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:38.202003 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7fb1f-852a-4df0-a12b-5e53c2718301 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:38.202266 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7fb1f-852a-4df0-a12b-5e53c2718301 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:38.203875 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-0bc7fb1f-852a-4df0-a12b-5e53c2718301 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:38.283386 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:38.316789 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc7fb1f-852a-4df0-a12b-5e53c2718301 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:38.318276 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc7fb1f-852a-4df0-a12b-5e53c2718301 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:38.337693 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 670/1329] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 14:59:38 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 14:59:40.722272 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2220f-bd6c-47ef-9d52-411612f3cf6e None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:40.722841 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2220f-bd6c-47ef-9d52-411612f3cf6e None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:40.722841 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2220f-bd6c-47ef-9d52-411612f3cf6e None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:40.724191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 660/1330] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:59:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:40.729476 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0638d8e-d197-4119-bb47-3152c1b35e5e None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:40.729908 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0638d8e-d197-4119-bb47-3152c1b35e5e None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:40.730023 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0638d8e-d197-4119-bb47-3152c1b35e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:40.795153 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-b0638d8e-d197-4119-bb47-3152c1b35e5e None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:40.796364 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-b0638d8e-d197-4119-bb47-3152c1b35e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:40.891603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 671/1331] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 14:59:41.172446 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2220f-bd6c-47ef-9d52-411612f3cf6e None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:41.173972 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-81c2220f-bd6c-47ef-9d52-411612f3cf6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:41.276639 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2220f-bd6c-47ef-9d52-411612f3cf6e None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:41.349479 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34855d3-bee1-421c-88a9-d2458149dffa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6eb236e80b4395a7aa77020b2f3a4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:41.350081 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34855d3-bee1-421c-88a9-d2458149dffa service neutron] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:41.350361 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34855d3-bee1-421c-88a9-d2458149dffa service neutron] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:41.350632 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34855d3-bee1-421c-88a9-d2458149dffa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:41.353189 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-a34855d3-bee1-421c-88a9-d2458149dffa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:41.380324 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:41.461840 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34855d3-bee1-421c-88a9-d2458149dffa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:41.463323 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34855d3-bee1-421c-88a9-d2458149dffa service neutron] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:41.481594 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 661/1332] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 14:59:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 14:59:43.682166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9e8de-5f64-478e-9a90-78d056c3bede None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:43.682736 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9e8de-5f64-478e-9a90-78d056c3bede None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:43.682736 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9e8de-5f64-478e-9a90-78d056c3bede None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:43.684127 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 672/1333] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:43.689490 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb71dd00-8e21-43f7-8ce0-0c8f82d38e46 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:43.689875 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb71dd00-8e21-43f7-8ce0-0c8f82d38e46 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:43.690033 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb71dd00-8e21-43f7-8ce0-0c8f82d38e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:43.755538 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-cb71dd00-8e21-43f7-8ce0-0c8f82d38e46 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:43.756751 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-cb71dd00-8e21-43f7-8ce0-0c8f82d38e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:43.831502 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:43.855463 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 662/1334] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 14:59:44.142717 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9e8de-5f64-478e-9a90-78d056c3bede None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:44.144350 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-74b9e8de-5f64-478e-9a90-78d056c3bede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:44.184267 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:44.249722 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9e8de-5f64-478e-9a90-78d056c3bede None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:44.325257 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc5a05-e597-47fa-a4bd-5bbc7763ebd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6eb236e80b4395a7aa77020b2f3a4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:44.325871 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc5a05-e597-47fa-a4bd-5bbc7763ebd5 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:44.326143 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc5a05-e597-47fa-a4bd-5bbc7763ebd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:44.326409 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc5a05-e597-47fa-a4bd-5bbc7763ebd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:44.328020 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-12dc5a05-e597-47fa-a4bd-5bbc7763ebd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:44.410127 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:44.436725 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dc5a05-e597-47fa-a4bd-5bbc7763ebd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:44.438429 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dc5a05-e597-47fa-a4bd-5bbc7763ebd5 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:44.456339 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 673/1335] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 14:59:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 14:59:46.940772 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad34aa3-ddb9-4dcf-88d7-6fcae4403fe3 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:46.941287 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad34aa3-ddb9-4dcf-88d7-6fcae4403fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:46.941401 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad34aa3-ddb9-4dcf-88d7-6fcae4403fe3 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:46.942658 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 663/1336] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:59:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:46.948523 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fbd8c3-ea85-44b7-9597-988684a1dd45 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:46.948937 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fbd8c3-ea85-44b7-9597-988684a1dd45 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:46.949037 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fbd8c3-ea85-44b7-9597-988684a1dd45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:47.015155 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-a9fbd8c3-ea85-44b7-9597-988684a1dd45 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:47.016411 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a9fbd8c3-ea85-44b7-9597-988684a1dd45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:47.110796 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 674/1337] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 14:59:47.397718 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad34aa3-ddb9-4dcf-88d7-6fcae4403fe3 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:47.399403 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-3ad34aa3-ddb9-4dcf-88d7-6fcae4403fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:47.501759 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad34aa3-ddb9-4dcf-88d7-6fcae4403fe3 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:47.574121 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50d659-ce0c-4077-b6de-890b089c81e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6eb236e80b4395a7aa77020b2f3a4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:47.574734 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50d659-ce0c-4077-b6de-890b089c81e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:47.575041 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50d659-ce0c-4077-b6de-890b089c81e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:47.575311 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50d659-ce0c-4077-b6de-890b089c81e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:47.576888 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-4a50d659-ce0c-4077-b6de-890b089c81e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:47.613818 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:47.687986 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a50d659-ce0c-4077-b6de-890b089c81e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:47.689357 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a50d659-ce0c-4077-b6de-890b089c81e5 service neutron] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:47.707750 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 664/1338] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 14:59:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 14:59:51.767693 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2388287-4091-4a1a-9d32-d9672b2d3f89 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:51.768224 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2388287-4091-4a1a-9d32-d9672b2d3f89 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:51.768224 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2388287-4091-4a1a-9d32-d9672b2d3f89 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:51.769481 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 675/1339] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:51.774794 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b8255-d92b-4df5-92c0-39e8afa17ff0 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:51.775266 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b8255-d92b-4df5-92c0-39e8afa17ff0 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:51.775371 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b8255-d92b-4df5-92c0-39e8afa17ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:51.842225 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-781b8255-d92b-4df5-92c0-39e8afa17ff0 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:51.843380 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-781b8255-d92b-4df5-92c0-39e8afa17ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:51.930676 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:51.942185 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 665/1340] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 14:59:52.236464 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2388287-4091-4a1a-9d32-d9672b2d3f89 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:52.238086 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-e2388287-4091-4a1a-9d32-d9672b2d3f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:52.288713 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:52.346168 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2388287-4091-4a1a-9d32-d9672b2d3f89 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:52.421543 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46020e9d-b242-4da0-9f38-c2b829cfab09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6eb236e80b4395a7aa77020b2f3a4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:52.422155 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46020e9d-b242-4da0-9f38-c2b829cfab09 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:52.422429 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46020e9d-b242-4da0-9f38-c2b829cfab09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:52.422694 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46020e9d-b242-4da0-9f38-c2b829cfab09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:52.424295 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-46020e9d-b242-4da0-9f38-c2b829cfab09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:52.519453 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:52.535966 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46020e9d-b242-4da0-9f38-c2b829cfab09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:52.537411 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46020e9d-b242-4da0-9f38-c2b829cfab09 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:52.557321 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 676/1341] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 14:59:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 14:59:55.391931 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05fbb7a-7ff2-4b14-b076-87fadb5f9dea None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:55.392498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05fbb7a-7ff2-4b14-b076-87fadb5f9dea None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:55.392498 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05fbb7a-7ff2-4b14-b076-87fadb5f9dea None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:55.393830 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 666/1342] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 14:59:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:55.399786 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb68071-1294-4ae0-886b-d8053be1e94f None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:55.400241 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb68071-1294-4ae0-886b-d8053be1e94f None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:55.400343 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb68071-1294-4ae0-886b-d8053be1e94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:55.467257 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-0fb68071-1294-4ae0-886b-d8053be1e94f None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:55.468487 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-0fb68071-1294-4ae0-886b-d8053be1e94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:55.565761 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 677/1343] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 14:59:55.859714 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05fbb7a-7ff2-4b14-b076-87fadb5f9dea None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:55.861279 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-c05fbb7a-7ff2-4b14-b076-87fadb5f9dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:55.979782 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05fbb7a-7ff2-4b14-b076-87fadb5f9dea None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:56.056525 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd84e9ae-0334-41fe-8e13-78baa9c890ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6eb236e80b4395a7aa77020b2f3a4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:56.057129 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84e9ae-0334-41fe-8e13-78baa9c890ce service neutron] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:56.057403 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84e9ae-0334-41fe-8e13-78baa9c890ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:56.057717 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84e9ae-0334-41fe-8e13-78baa9c890ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:56.059270 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-dd84e9ae-0334-41fe-8e13-78baa9c890ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:56.103281 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:56.172147 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd84e9ae-0334-41fe-8e13-78baa9c890ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:56.173606 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd84e9ae-0334-41fe-8e13-78baa9c890ce service neutron] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:56.193574 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 667/1344] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 14:59:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:00:00.353015 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3440a417-66a1-4c0b-9dd7-58a305987f24 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:00.353522 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3440a417-66a1-4c0b-9dd7-58a305987f24 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:00.353606 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3440a417-66a1-4c0b-9dd7-58a305987f24 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:00.355241 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 678/1345] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 15:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:00.360818 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4fd01-7772-44cc-9d57-7906b7c50416 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:00.361337 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4fd01-7772-44cc-9d57-7906b7c50416 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:00.361337 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4fd01-7772-44cc-9d57-7906b7c50416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:00.440906 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-c9e4fd01-7772-44cc-9d57-7906b7c50416 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:00.442284 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-c9e4fd01-7772-44cc-9d57-7906b7c50416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:00.560347 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:00.562773 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 668/1346] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:00:00.852380 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3440a417-66a1-4c0b-9dd7-58a305987f24 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:00.853914 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-3440a417-66a1-4c0b-9dd7-58a305987f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:00.911956 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:00.961868 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3440a417-66a1-4c0b-9dd7-58a305987f24 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:01.039207 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c9864-6897-4511-91f9-8b3af7265ccc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6eb236e80b4395a7aa77020b2f3a4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:01.039822 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c9864-6897-4511-91f9-8b3af7265ccc service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:01.040106 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c9864-6897-4511-91f9-8b3af7265ccc service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:01.040372 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c9864-6897-4511-91f9-8b3af7265ccc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:01.041973 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-c98c9864-6897-4511-91f9-8b3af7265ccc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:01.152492 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:01.156262 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c9864-6897-4511-91f9-8b3af7265ccc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:01.158432 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c9864-6897-4511-91f9-8b3af7265ccc service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:01.178186 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 679/1347] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:00:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:00:06.079155 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f2e5b-323f-41b8-bb51-82bf57f1f208 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:06.079155 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f2e5b-323f-41b8-bb51-82bf57f1f208 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:06.079632 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f2e5b-323f-41b8-bb51-82bf57f1f208 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:06.082417 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 669/1348] 10.222.0.33 () {54 vars in 921 bytes} [Sun Sep 11 15:00:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:06.086442 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac0803-d6c9-49c8-8c58-9dd653567c9c None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:06.087292 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac0803-d6c9-49c8-8c58-9dd653567c9c None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:06.089046 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac0803-d6c9-49c8-8c58-9dd653567c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:06.208962 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-52ac0803-d6c9-49c8-8c58-9dd653567c9c None None] MFA Rules not processed for user `9390c5960b404bedbead2becfdc6c7af`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:06.211579 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-52ac0803-d6c9-49c8-8c58-9dd653567c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:06.382195 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 680/1349] 10.222.0.33 () {58 vars in 1018 bytes} [Sun Sep 11 15:00:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:00:10.502882 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed496d-0754-4200-b768-1b4ae857467b None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:10.503453 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed496d-0754-4200-b768-1b4ae857467b None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:10.503453 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed496d-0754-4200-b768-1b4ae857467b None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:10.504780 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 670/1350] 10.222.0.33 () {54 vars in 921 bytes} [Sun Sep 11 15:00:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:10.510623 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaed2c2-3199-447d-8e03-6cfa1568ab74 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:10.511068 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaed2c2-3199-447d-8e03-6cfa1568ab74 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:10.511269 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaed2c2-3199-447d-8e03-6cfa1568ab74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:10.566717 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:10.592204 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-faaed2c2-3199-447d-8e03-6cfa1568ab74 None None] MFA Rules not processed for user `9390c5960b404bedbead2becfdc6c7af`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:10.593435 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-faaed2c2-3199-447d-8e03-6cfa1568ab74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:10.706674 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 681/1351] 10.222.0.33 () {58 vars in 1018 bytes} [Sun Sep 11 15:00:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:00:12.179766 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f1609-b4ca-48d2-95c9-b7099d9605ff None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:12.180343 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f1609-b4ca-48d2-95c9-b7099d9605ff None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:12.180343 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f1609-b4ca-48d2-95c9-b7099d9605ff None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:12.181930 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 671/1352] 10.222.0.33 () {54 vars in 921 bytes} [Sun Sep 11 15:00:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:12.186986 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2a621-4afa-40d6-8332-7e849838d37d None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:12.187711 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2a621-4afa-40d6-8332-7e849838d37d None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:12.187815 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2a621-4afa-40d6-8332-7e849838d37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:12.268327 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-ebe2a621-4afa-40d6-8332-7e849838d37d None None] MFA Rules not processed for user `9390c5960b404bedbead2becfdc6c7af`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:12.269486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-ebe2a621-4afa-40d6-8332-7e849838d37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:12.311652 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:12.376343 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 682/1353] 10.222.0.33 () {58 vars in 1018 bytes} [Sun Sep 11 15:00:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:00:16.103067 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9cd502-273d-4769-a96b-e8b3039bc232 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:16.104046 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9cd502-273d-4769-a96b-e8b3039bc232 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:16.104552 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9cd502-273d-4769-a96b-e8b3039bc232 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:16.106385 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 672/1354] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 15:00:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:16.113363 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278b6a2-8e8e-40cb-9636-cd5e2820382c None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:16.114296 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278b6a2-8e8e-40cb-9636-cd5e2820382c None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:16.114580 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278b6a2-8e8e-40cb-9636-cd5e2820382c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:16.192715 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-0278b6a2-8e8e-40cb-9636-cd5e2820382c None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:16.194073 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-0278b6a2-8e8e-40cb-9636-cd5e2820382c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:16.299615 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 683/1355] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:00:16.601617 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df478674-82bd-407a-ae33-e4ec986c4288 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:16.602168 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df478674-82bd-407a-ae33-e4ec986c4288 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:16.602611 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df478674-82bd-407a-ae33-e4ec986c4288 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:16.604622 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 673/1356] 10.222.0.33 () {54 vars in 962 bytes} [Sun Sep 11 15:00:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:16.613395 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7e2b0-3c73-4573-8e05-0a9695896dbc None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:16.613917 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7e2b0-3c73-4573-8e05-0a9695896dbc None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:16.614282 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7e2b0-3c73-4573-8e05-0a9695896dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:16.633819 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:16.703410 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-7da7e2b0-3c73-4573-8e05-0a9695896dbc None None] MFA Rules not processed for user `fc0ea93084b0482fb2dd6d802987942f`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:16.704605 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7da7e2b0-3c73-4573-8e05-0a9695896dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:16.829469 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 684/1357] 10.222.0.33 () {58 vars in 1059 bytes} [Sun Sep 11 15:00:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:00:16.837231 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df478674-82bd-407a-ae33-e4ec986c4288 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:16.838737 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-df478674-82bd-407a-ae33-e4ec986c4288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:16.997375 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df478674-82bd-407a-ae33-e4ec986c4288 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:17.074935 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:17.118486 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f3641-021e-46ec-ba42-b6953adf216c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc0ea93084b0482fb2dd6d802987942f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:17.119815 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f3641-021e-46ec-ba42-b6953adf216c service nova] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:17.120098 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f3641-021e-46ec-ba42-b6953adf216c service nova] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:17.120372 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f3641-021e-46ec-ba42-b6953adf216c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:17.121951 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-824f3641-021e-46ec-ba42-b6953adf216c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:17.216811 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f3641-021e-46ec-ba42-b6953adf216c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:17.218470 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f3641-021e-46ec-ba42-b6953adf216c service nova] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:17.239637 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 674/1358] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:00:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:00:19.687928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6fe3c9-1702-40d7-8d55-8f86b8b23fc4 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:19.688491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6fe3c9-1702-40d7-8d55-8f86b8b23fc4 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:19.688491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6fe3c9-1702-40d7-8d55-8f86b8b23fc4 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:19.689808 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 685/1359] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 15:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:19.695580 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220e4c6-e0b8-4cc6-aa12-82f746afe5ea None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:19.696005 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220e4c6-e0b8-4cc6-aa12-82f746afe5ea None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:19.696088 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220e4c6-e0b8-4cc6-aa12-82f746afe5ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:19.766450 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-9220e4c6-e0b8-4cc6-aa12-82f746afe5ea None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:19.767692 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-9220e4c6-e0b8-4cc6-aa12-82f746afe5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:19.813331 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:19.877329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 675/1360] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:00:20.165277 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6fe3c9-1702-40d7-8d55-8f86b8b23fc4 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:20.166822 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-4b6fe3c9-1702-40d7-8d55-8f86b8b23fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:20.238241 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:20.327704 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6fe3c9-1702-40d7-8d55-8f86b8b23fc4 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:20.467370 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a2281-3f60-42f0-85a2-6012da60b622 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc0ea93084b0482fb2dd6d802987942f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:20.468656 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a2281-3f60-42f0-85a2-6012da60b622 service nova] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:20.468944 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a2281-3f60-42f0-85a2-6012da60b622 service nova] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:20.469218 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a2281-3f60-42f0-85a2-6012da60b622 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:20.470789 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-3c4a2281-3f60-42f0-85a2-6012da60b622 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:20.496213 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:20.577883 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a2281-3f60-42f0-85a2-6012da60b622 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:20.579551 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a2281-3f60-42f0-85a2-6012da60b622 service nova] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:20.599742 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 686/1361] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:00:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:00:23.102587 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d1c3c2-2be9-4421-8ef0-e9e2dc021ce2 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:23.103175 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d1c3c2-2be9-4421-8ef0-e9e2dc021ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:23.103175 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d1c3c2-2be9-4421-8ef0-e9e2dc021ce2 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:23.104721 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 676/1362] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 15:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:23.110250 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c22d4-cd28-4993-a990-2e9a91dde43e None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:23.110730 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c22d4-cd28-4993-a990-2e9a91dde43e None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:23.110922 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c22d4-cd28-4993-a990-2e9a91dde43e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:23.184330 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-e86c22d4-cd28-4993-a990-2e9a91dde43e None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:23.185613 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-e86c22d4-cd28-4993-a990-2e9a91dde43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:23.294738 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:23.303137 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 687/1363] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:00:23.590061 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd1b8d-57f1-4ef7-b3b8-eaeb457e2d1d None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:23.590406 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd1b8d-57f1-4ef7-b3b8-eaeb457e2d1d None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:23.590619 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd1b8d-57f1-4ef7-b3b8-eaeb457e2d1d None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:23.592198 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 677/1364] 10.222.0.33 () {54 vars in 962 bytes} [Sun Sep 11 15:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:23.599071 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a26fb-898e-43dd-820f-682499d39cbe None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:23.599647 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a26fb-898e-43dd-820f-682499d39cbe None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:23.600536 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a26fb-898e-43dd-820f-682499d39cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:23.681515 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-091a26fb-898e-43dd-820f-682499d39cbe None None] MFA Rules not processed for user `fc0ea93084b0482fb2dd6d802987942f`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:23.682904 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-091a26fb-898e-43dd-820f-682499d39cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:23.805422 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 688/1365] 10.222.0.33 () {58 vars in 1059 bytes} [Sun Sep 11 15:00:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:00:23.814133 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd1b8d-57f1-4ef7-b3b8-eaeb457e2d1d None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:23.815694 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-abcd1b8d-57f1-4ef7-b3b8-eaeb457e2d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:23.974941 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd1b8d-57f1-4ef7-b3b8-eaeb457e2d1d None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:24.001986 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:24.105363 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536409bc-3341-49f1-948a-d02259219097 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc0ea93084b0482fb2dd6d802987942f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:24.105991 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536409bc-3341-49f1-948a-d02259219097 service nova] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:24.106252 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536409bc-3341-49f1-948a-d02259219097 service nova] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:24.106527 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536409bc-3341-49f1-948a-d02259219097 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:24.108547 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-536409bc-3341-49f1-948a-d02259219097 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:24.205643 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536409bc-3341-49f1-948a-d02259219097 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:24.207299 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536409bc-3341-49f1-948a-d02259219097 service nova] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:24.227335 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 678/1366] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:00:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:00:24.446486 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc6493-1b8b-451e-87bd-412b6757ba51 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:24.447124 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc6493-1b8b-451e-87bd-412b6757ba51 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:24.447124 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc6493-1b8b-451e-87bd-412b6757ba51 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:24.448607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 689/1367] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 15:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:24.454131 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fde0a8-b05b-4993-b6c4-5b79590cf125 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:24.454463 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fde0a8-b05b-4993-b6c4-5b79590cf125 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:24.454695 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fde0a8-b05b-4993-b6c4-5b79590cf125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:24.491847 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:24.543340 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-a5fde0a8-b05b-4993-b6c4-5b79590cf125 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:24.544593 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-a5fde0a8-b05b-4993-b6c4-5b79590cf125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:24.662903 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 679/1368] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:00:25.010383 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c8566-f61a-48f1-bdb0-a6b960d675a7 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:25.010383 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c8566-f61a-48f1-bdb0-a6b960d675a7 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:25.010383 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c8566-f61a-48f1-bdb0-a6b960d675a7 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:25.012571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 690/1369] 10.222.0.33 () {54 vars in 964 bytes} [Sun Sep 11 15:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:25.018131 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca26794-1bc1-4a1f-922e-b9914345a287 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:25.018131 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca26794-1bc1-4a1f-922e-b9914345a287 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:25.018131 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca26794-1bc1-4a1f-922e-b9914345a287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:25.106275 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-4ca26794-1bc1-4a1f-922e-b9914345a287 None None] MFA Rules not processed for user `c793eafcdba249a5af3ab948e9774367`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:25.107666 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-4ca26794-1bc1-4a1f-922e-b9914345a287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:25.157928 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:25.194993 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 680/1370] 10.222.0.33 () {58 vars in 1061 bytes} [Sun Sep 11 15:00:25 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:00:25.206291 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c8566-f61a-48f1-bdb0-a6b960d675a7 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:25.206291 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-661c8566-f61a-48f1-bdb0-a6b960d675a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:25.257713 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:25.319587 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c8566-f61a-48f1-bdb0-a6b960d675a7 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:25.407155 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5cc9cb-b31e-4e7d-8bd3-d3c06b05ba04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c793eafcdba249a5af3ab948e9774367', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:25.407923 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cc9cb-b31e-4e7d-8bd3-d3c06b05ba04 service cinder] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:25.408070 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cc9cb-b31e-4e7d-8bd3-d3c06b05ba04 service cinder] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:25.408378 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cc9cb-b31e-4e7d-8bd3-d3c06b05ba04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:25.409851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-4b5cc9cb-b31e-4e7d-8bd3-d3c06b05ba04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:25.518100 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:25.533968 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5cc9cb-b31e-4e7d-8bd3-d3c06b05ba04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:25.535244 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5cc9cb-b31e-4e7d-8bd3-d3c06b05ba04 service cinder] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:25.559374 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 691/1371] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:00:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:00:26.488590 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b58a5e-bab8-49f8-8c95-326ecec0bfbb None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:26.489127 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b58a5e-bab8-49f8-8c95-326ecec0bfbb None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:26.489127 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b58a5e-bab8-49f8-8c95-326ecec0bfbb None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:26.491458 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 681/1372] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 15:00:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:26.497203 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb44cf2-390c-4c8c-95a7-7c53d1122d9b None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:26.497442 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb44cf2-390c-4c8c-95a7-7c53d1122d9b None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:26.497749 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb44cf2-390c-4c8c-95a7-7c53d1122d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:26.565650 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-6cb44cf2-390c-4c8c-95a7-7c53d1122d9b None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:26.566895 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6cb44cf2-390c-4c8c-95a7-7c53d1122d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:26.665070 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 692/1373] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:00:26.965085 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b58a5e-bab8-49f8-8c95-326ecec0bfbb None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:26.967107 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-73b58a5e-bab8-49f8-8c95-326ecec0bfbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:27.145540 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b58a5e-bab8-49f8-8c95-326ecec0bfbb None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:27.204889 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:27.271340 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc5481c-2046-4bee-abcf-9a5db5e9f236 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc0ea93084b0482fb2dd6d802987942f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:27.272228 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5481c-2046-4bee-abcf-9a5db5e9f236 service nova] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:27.272506 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5481c-2046-4bee-abcf-9a5db5e9f236 service nova] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:27.272781 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5481c-2046-4bee-abcf-9a5db5e9f236 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:27.274414 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-bdc5481c-2046-4bee-abcf-9a5db5e9f236 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:27.373775 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc5481c-2046-4bee-abcf-9a5db5e9f236 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:27.375446 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc5481c-2046-4bee-abcf-9a5db5e9f236 service nova] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:27.395125 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 682/1374] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:00:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:00:29.652418 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba5dea-8cf0-48f3-8117-02a44365ffc3 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:29.652948 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba5dea-8cf0-48f3-8117-02a44365ffc3 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:29.652948 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba5dea-8cf0-48f3-8117-02a44365ffc3 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:29.654408 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 693/1375] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 15:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:29.659934 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267e253-764f-4d95-a505-472f24e30712 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:29.660431 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267e253-764f-4d95-a505-472f24e30712 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:29.660518 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267e253-764f-4d95-a505-472f24e30712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:29.733924 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-a267e253-764f-4d95-a505-472f24e30712 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:29.735183 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-a267e253-764f-4d95-a505-472f24e30712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:29.743832 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:29.844273 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 683/1376] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:00:30.136717 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ba5dea-8cf0-48f3-8117-02a44365ffc3 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:30.138281 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-27ba5dea-8cf0-48f3-8117-02a44365ffc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:30.203728 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:30.301369 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ba5dea-8cf0-48f3-8117-02a44365ffc3 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:30.429769 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9035ce0-fd32-430f-bdb3-238a83ef17ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc0ea93084b0482fb2dd6d802987942f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:30.430384 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9035ce0-fd32-430f-bdb3-238a83ef17ad service nova] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:30.430671 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9035ce0-fd32-430f-bdb3-238a83ef17ad service nova] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:30.430983 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9035ce0-fd32-430f-bdb3-238a83ef17ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:30.432704 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-e9035ce0-fd32-430f-bdb3-238a83ef17ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:30.454007 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:30.535358 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9035ce0-fd32-430f-bdb3-238a83ef17ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:30.537327 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9035ce0-fd32-430f-bdb3-238a83ef17ad service nova] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:30.557670 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 694/1377] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:00:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:00:32.807948 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e01698-94ba-44a5-b341-0a372a51cf84 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:32.808496 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e01698-94ba-44a5-b341-0a372a51cf84 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:32.808496 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e01698-94ba-44a5-b341-0a372a51cf84 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:32.810128 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 684/1378] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 15:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:32.815886 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a26c2-75db-4539-8ca0-510dbd4e5642 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:32.816357 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a26c2-75db-4539-8ca0-510dbd4e5642 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:32.816542 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a26c2-75db-4539-8ca0-510dbd4e5642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:32.885960 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-3b7a26c2-75db-4539-8ca0-510dbd4e5642 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:32.887311 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-3b7a26c2-75db-4539-8ca0-510dbd4e5642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:33.001199 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:33.010945 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 695/1379] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:00:33.308187 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e01698-94ba-44a5-b341-0a372a51cf84 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:33.309648 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-a9e01698-94ba-44a5-b341-0a372a51cf84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:33.451865 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:33.468206 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e01698-94ba-44a5-b341-0a372a51cf84 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:33.612607 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab98c1-a9ac-4457-9c46-b841bd578a00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc0ea93084b0482fb2dd6d802987942f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:33.613139 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab98c1-a9ac-4457-9c46-b841bd578a00 service nova] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:33.613421 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab98c1-a9ac-4457-9c46-b841bd578a00 service nova] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:33.613691 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab98c1-a9ac-4457-9c46-b841bd578a00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:33.615341 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-a4ab98c1-a9ac-4457-9c46-b841bd578a00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:33.709031 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ab98c1-a9ac-4457-9c46-b841bd578a00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:33.710938 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ab98c1-a9ac-4457-9c46-b841bd578a00 service nova] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:33.721068 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:33.730072 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 685/1380] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:00:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:00:35.953867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb3e78-4c05-41c4-8982-b65cf70fb368 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:35.954491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb3e78-4c05-41c4-8982-b65cf70fb368 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:35.954491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb3e78-4c05-41c4-8982-b65cf70fb368 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:35.955762 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 696/1381] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 15:00:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:35.962234 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ab6ca-1869-445e-a0d8-ca2bcc995bfa None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:35.962777 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ab6ca-1869-445e-a0d8-ca2bcc995bfa None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:35.962777 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ab6ca-1869-445e-a0d8-ca2bcc995bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:36.045647 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-848ab6ca-1869-445e-a0d8-ca2bcc995bfa None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:36.047155 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-848ab6ca-1869-445e-a0d8-ca2bcc995bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:36.161276 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 686/1382] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:00:36.500037 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb3e78-4c05-41c4-8982-b65cf70fb368 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:36.501594 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-82cb3e78-4c05-41c4-8982-b65cf70fb368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:36.695034 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb3e78-4c05-41c4-8982-b65cf70fb368 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:36.799845 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:36.850509 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e7b180-5a26-4da6-be6c-ccdc57829625 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc0ea93084b0482fb2dd6d802987942f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:36.851156 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7b180-5a26-4da6-be6c-ccdc57829625 service nova] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:36.851438 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7b180-5a26-4da6-be6c-ccdc57829625 service nova] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:36.851713 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7b180-5a26-4da6-be6c-ccdc57829625 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:36.853331 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-99e7b180-5a26-4da6-be6c-ccdc57829625 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:36.975643 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e7b180-5a26-4da6-be6c-ccdc57829625 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:36.977544 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e7b180-5a26-4da6-be6c-ccdc57829625 service nova] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:37.001826 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 697/1383] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:00:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:00:39.196348 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22f2ea-87fc-4efb-b8c2-85264b52a79e None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:39.196899 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22f2ea-87fc-4efb-b8c2-85264b52a79e None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:39.196899 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22f2ea-87fc-4efb-b8c2-85264b52a79e None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:39.198377 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 687/1384] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 15:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:39.203491 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3706d0d7-acef-425c-bf92-1954dd987d3b None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:39.203983 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3706d0d7-acef-425c-bf92-1954dd987d3b None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:39.204101 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3706d0d7-acef-425c-bf92-1954dd987d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:39.272210 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-3706d0d7-acef-425c-bf92-1954dd987d3b None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:39.273427 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-3706d0d7-acef-425c-bf92-1954dd987d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:39.308156 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:39.374575 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 698/1385] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:00:39.664412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce22f2ea-87fc-4efb-b8c2-85264b52a79e None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:39.665925 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-ce22f2ea-87fc-4efb-b8c2-85264b52a79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:39.741422 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:39.811204 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce22f2ea-87fc-4efb-b8c2-85264b52a79e None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:39.924534 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d1438-3ba4-436a-9ca4-8a122f877f67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc0ea93084b0482fb2dd6d802987942f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:39.925054 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d1438-3ba4-436a-9ca4-8a122f877f67 service nova] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:39.925330 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d1438-3ba4-436a-9ca4-8a122f877f67 service nova] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:39.925598 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d1438-3ba4-436a-9ca4-8a122f877f67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:39.927819 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-461d1438-3ba4-436a-9ca4-8a122f877f67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:39.968815 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:40.025980 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d1438-3ba4-436a-9ca4-8a122f877f67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:40.027656 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d1438-3ba4-436a-9ca4-8a122f877f67 service nova] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:40.046113 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 688/1386] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:00:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:00:40.902865 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870713c-49bf-4cc4-b975-70858bcffa09 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:40.903415 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870713c-49bf-4cc4-b975-70858bcffa09 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:40.903415 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870713c-49bf-4cc4-b975-70858bcffa09 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:40.904774 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 699/1387] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 15:00:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:40.909944 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd49c8-996b-433d-94f2-2c71a16711fd None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:40.910375 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd49c8-996b-433d-94f2-2c71a16711fd None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:40.910553 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd49c8-996b-433d-94f2-2c71a16711fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:40.984841 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-e4fd49c8-996b-433d-94f2-2c71a16711fd None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:40.986062 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-e4fd49c8-996b-433d-94f2-2c71a16711fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:41.105805 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 689/1388] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:00:41.408621 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b508ab-006d-48d5-a4f2-52a7abf66cd6 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:41.408918 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b508ab-006d-48d5-a4f2-52a7abf66cd6 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:41.409196 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b508ab-006d-48d5-a4f2-52a7abf66cd6 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:41.410755 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 700/1389] 10.222.0.33 () {54 vars in 966 bytes} [Sun Sep 11 15:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:41.416257 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad048cb-9d4b-4a86-a8ba-7daab10562b2 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:41.416539 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad048cb-9d4b-4a86-a8ba-7daab10562b2 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:41.416816 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad048cb-9d4b-4a86-a8ba-7daab10562b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:41.432926 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:41.513034 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-7ad048cb-9d4b-4a86-a8ba-7daab10562b2 None None] MFA Rules not processed for user `cd9c8b53871e415facea40c6694f4171`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:41.514245 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-7ad048cb-9d4b-4a86-a8ba-7daab10562b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:41.594990 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 690/1390] 10.222.0.33 () {58 vars in 1063 bytes} [Sun Sep 11 15:00:41 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:00:41.605385 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b508ab-006d-48d5-a4f2-52a7abf66cd6 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:41.606944 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-21b508ab-006d-48d5-a4f2-52a7abf66cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:41.712867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b508ab-006d-48d5-a4f2-52a7abf66cd6 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:41.772927 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:41.789504 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40c7f1-381d-4194-aed3-c66b14d32525 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd9c8b53871e415facea40c6694f4171', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:41.790111 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40c7f1-381d-4194-aed3-c66b14d32525 service glance] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:41.790387 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40c7f1-381d-4194-aed3-c66b14d32525 service glance] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:41.790653 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40c7f1-381d-4194-aed3-c66b14d32525 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:41.792264 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-ac40c7f1-381d-4194-aed3-c66b14d32525 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:41.907770 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac40c7f1-381d-4194-aed3-c66b14d32525 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:41.909216 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac40c7f1-381d-4194-aed3-c66b14d32525 service glance] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:41.928690 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 701/1391] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:00:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:00:42.263633 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9be6a9-a3d2-4851-b8cb-6a62dd61bd20 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:42.264170 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9be6a9-a3d2-4851-b8cb-6a62dd61bd20 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:42.264170 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9be6a9-a3d2-4851-b8cb-6a62dd61bd20 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:42.265535 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 691/1392] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 15:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:42.271219 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acded48-a17e-4728-85b1-1cef144a4678 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:42.271532 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acded48-a17e-4728-85b1-1cef144a4678 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:42.271810 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acded48-a17e-4728-85b1-1cef144a4678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:42.352346 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-5acded48-a17e-4728-85b1-1cef144a4678 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:42.353599 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-5acded48-a17e-4728-85b1-1cef144a4678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:42.395953 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:42.454725 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 702/1393] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:00:42.676295 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c2914-0627-4e56-9efd-7fb8c496c94a None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:42.676586 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c2914-0627-4e56-9efd-7fb8c496c94a None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:42.676867 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c2914-0627-4e56-9efd-7fb8c496c94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:42.746123 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acded48-a17e-4728-85b1-1cef144a4678 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:42.747438 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-4e4c2914-0627-4e56-9efd-7fb8c496c94a None None] MFA Rules not processed for user `0aafb2bbba1d466fb22bb0824a531ddc`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:42.747567 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-5acded48-a17e-4728-85b1-1cef144a4678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:42.748638 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-4e4c2914-0627-4e56-9efd-7fb8c496c94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:42.804157 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:42.837424 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 692/1394] 10.222.0.33 () {58 vars in 1009 bytes} [Sun Sep 11 15:00:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:00:42.869941 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acded48-a17e-4728-85b1-1cef144a4678 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:42.880044 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7739e90c-47ee-410a-834b-f9a8ded6b751 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:42.881090 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7739e90c-47ee-410a-834b-f9a8ded6b751 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:42.881385 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7739e90c-47ee-410a-834b-f9a8ded6b751 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:42.882798 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 693/1395] 10.222.0.33 () {54 vars in 943 bytes} [Sun Sep 11 15:00:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 15:00:42.888758 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727d061-88a7-444c-b5a9-b8749e1a0ffa None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:42.889115 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727d061-88a7-444c-b5a9-b8749e1a0ffa None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:42.889478 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727d061-88a7-444c-b5a9-b8749e1a0ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:42.929620 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:42.969411 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-f727d061-88a7-444c-b5a9-b8749e1a0ffa None None] MFA Rules not processed for user `698f1293294d45b89e04d85649f454d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:42.970622 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-f727d061-88a7-444c-b5a9-b8749e1a0ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:42.989945 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dcc557-e411-4490-ac6e-1bd7667d244b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc0ea93084b0482fb2dd6d802987942f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:42.990704 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dcc557-e411-4490-ac6e-1bd7667d244b service nova] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:42.991015 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dcc557-e411-4490-ac6e-1bd7667d244b service nova] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:42.991289 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dcc557-e411-4490-ac6e-1bd7667d244b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:42.992904 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-17dcc557-e411-4490-ac6e-1bd7667d244b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:43.081672 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 694/1396] 10.222.0.33 () {58 vars in 1040 bytes} [Sun Sep 11 15:00:42 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:00:43.087612 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dcc557-e411-4490-ac6e-1bd7667d244b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:43.089232 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dcc557-e411-4490-ac6e-1bd7667d244b service nova] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:43.091855 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f727d061-88a7-444c-b5a9-b8749e1a0ffa None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:43.102144 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f727d061-88a7-444c-b5a9-b8749e1a0ffa None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:43.108323 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 703/1397] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:00:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:00:43.134685 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:43.225586 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f4022-34e2-4331-9c18-22f103f58ca1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698f1293294d45b89e04d85649f454d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:43.226213 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f4022-34e2-4331-9c18-22f103f58ca1 service swift] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:43.226492 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f4022-34e2-4331-9c18-22f103f58ca1 service swift] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:43.226760 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f4022-34e2-4331-9c18-22f103f58ca1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:43.228530 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-128f4022-34e2-4331-9c18-22f103f58ca1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:43.322216 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128f4022-34e2-4331-9c18-22f103f58ca1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:43.323956 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128f4022-34e2-4331-9c18-22f103f58ca1 service swift] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:43.332984 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 695/1398] 10.222.0.33 () {60 vars in 1384 bytes} [Sun Sep 11 15:00:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 11 15:00:45.327021 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc0938-eff5-49db-bd4d-bc79a2da1b7c None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:45.327575 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc0938-eff5-49db-bd4d-bc79a2da1b7c None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:45.327575 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc0938-eff5-49db-bd4d-bc79a2da1b7c None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:45.328914 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 704/1399] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 15:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:45.334907 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104d1c8-4f1d-4723-a6d5-f68e5a4f2a69 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:45.335359 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104d1c8-4f1d-4723-a6d5-f68e5a4f2a69 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:45.335591 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104d1c8-4f1d-4723-a6d5-f68e5a4f2a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:45.411721 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-7104d1c8-4f1d-4723-a6d5-f68e5a4f2a69 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:45.413457 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-7104d1c8-4f1d-4723-a6d5-f68e5a4f2a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:45.420458 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:45.531617 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 696/1400] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:00:45.829337 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc0938-eff5-49db-bd4d-bc79a2da1b7c None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:45.830494 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-92bc0938-eff5-49db-bd4d-bc79a2da1b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:45.906464 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:46.001846 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc0938-eff5-49db-bd4d-bc79a2da1b7c None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:46.133003 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f525fdb2-e6c3-4463-9120-d0cf15dcc78d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc0ea93084b0482fb2dd6d802987942f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:46.133639 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f525fdb2-e6c3-4463-9120-d0cf15dcc78d service nova] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:46.134018 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f525fdb2-e6c3-4463-9120-d0cf15dcc78d service nova] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:46.134357 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f525fdb2-e6c3-4463-9120-d0cf15dcc78d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:46.136080 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-f525fdb2-e6c3-4463-9120-d0cf15dcc78d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:46.165380 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:46.276936 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f525fdb2-e6c3-4463-9120-d0cf15dcc78d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:46.280753 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f525fdb2-e6c3-4463-9120-d0cf15dcc78d service nova] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:46.310850 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 705/1401] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:00:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:00:48.608906 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947004ab-2e71-4f0a-9d8b-0d314d064d3d None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:48.609507 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947004ab-2e71-4f0a-9d8b-0d314d064d3d None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:48.609507 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947004ab-2e71-4f0a-9d8b-0d314d064d3d None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:48.610835 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 697/1402] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 15:00:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:48.616492 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82cf0e-b451-4afb-bbd5-9cbf3523da91 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:48.620069 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82cf0e-b451-4afb-bbd5-9cbf3523da91 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:48.620069 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82cf0e-b451-4afb-bbd5-9cbf3523da91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:48.693018 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-0b82cf0e-b451-4afb-bbd5-9cbf3523da91 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:48.694260 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-0b82cf0e-b451-4afb-bbd5-9cbf3523da91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:48.804514 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 706/1403] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:00:49.094477 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947004ab-2e71-4f0a-9d8b-0d314d064d3d None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:49.096155 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-947004ab-2e71-4f0a-9d8b-0d314d064d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:49.260757 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:49.273485 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947004ab-2e71-4f0a-9d8b-0d314d064d3d None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:49.408606 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7578948-2ce4-4d25-aeda-f7909eb828ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc0ea93084b0482fb2dd6d802987942f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:49.409219 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7578948-2ce4-4d25-aeda-f7909eb828ef service nova] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:49.409501 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7578948-2ce4-4d25-aeda-f7909eb828ef service nova] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:49.409776 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7578948-2ce4-4d25-aeda-f7909eb828ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:49.411971 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-c7578948-2ce4-4d25-aeda-f7909eb828ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:49.517670 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7578948-2ce4-4d25-aeda-f7909eb828ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:49.519249 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7578948-2ce4-4d25-aeda-f7909eb828ef service nova] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:49.532345 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:49.544157 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 698/1404] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:00:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:00:51.760980 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81818955-bea1-445f-8a2b-a7ca9590c4a9 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:51.761587 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81818955-bea1-445f-8a2b-a7ca9590c4a9 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:51.761587 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81818955-bea1-445f-8a2b-a7ca9590c4a9 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:51.767163 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 707/1405] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 15:00:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:51.768940 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d6009-1fa6-459b-a52a-6e268e63673b None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:51.769403 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d6009-1fa6-459b-a52a-6e268e63673b None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:51.769502 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d6009-1fa6-459b-a52a-6e268e63673b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:51.847720 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-ea0d6009-1fa6-459b-a52a-6e268e63673b None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:51.849619 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-ea0d6009-1fa6-459b-a52a-6e268e63673b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:51.961054 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 699/1406] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:00:52.259412 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81818955-bea1-445f-8a2b-a7ca9590c4a9 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:52.261988 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-81818955-bea1-445f-8a2b-a7ca9590c4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:52.264428 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:52.427129 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81818955-bea1-445f-8a2b-a7ca9590c4a9 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:52.515617 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:52.563717 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28feb70-51a3-4e79-89cd-bb37ef2f0b61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc0ea93084b0482fb2dd6d802987942f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:52.564447 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28feb70-51a3-4e79-89cd-bb37ef2f0b61 service nova] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:52.564823 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28feb70-51a3-4e79-89cd-bb37ef2f0b61 service nova] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:52.565169 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28feb70-51a3-4e79-89cd-bb37ef2f0b61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:52.567022 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-e28feb70-51a3-4e79-89cd-bb37ef2f0b61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:52.678906 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28feb70-51a3-4e79-89cd-bb37ef2f0b61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:52.680693 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28feb70-51a3-4e79-89cd-bb37ef2f0b61 service nova] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:52.702650 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 708/1407] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:00:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:01:01.959433 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=67 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:02.699681 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=61 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:06.354385 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8cfa7-9827-4175-9f16-1fb9457ce4a7 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:06.354990 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8cfa7-9827-4175-9f16-1fb9457ce4a7 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:06.354990 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8cfa7-9827-4175-9f16-1fb9457ce4a7 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:06.356223 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 700/1408] 10.222.0.33 () {54 vars in 918 bytes} [Sun Sep 11 15:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:01:06.361293 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51b3b6-c651-438c-99ca-e3f1421a03df None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:06.361722 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51b3b6-c651-438c-99ca-e3f1421a03df None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:06.361839 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51b3b6-c651-438c-99ca-e3f1421a03df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:06.437684 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-ec51b3b6-c651-438c-99ca-e3f1421a03df None None] MFA Rules not processed for user `9390c5960b404bedbead2becfdc6c7af`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:01:06.438933 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-ec51b3b6-c651-438c-99ca-e3f1421a03df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:06.545713 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 709/1409] 10.222.0.33 () {58 vars in 1015 bytes} [Sun Sep 11 15:01:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:01:06.560191 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58681b3a-f02c-4951-aadb-0f239a2511e7 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:06.560480 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58681b3a-f02c-4951-aadb-0f239a2511e7 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:06.560757 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58681b3a-f02c-4951-aadb-0f239a2511e7 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:06.562086 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 701/1410] 10.222.0.33 () {54 vars in 918 bytes} [Sun Sep 11 15:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:01:06.567502 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d6dbe-5124-43af-8e10-1ed9805b5d29 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:06.567806 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d6dbe-5124-43af-8e10-1ed9805b5d29 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:06.568057 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d6dbe-5124-43af-8e10-1ed9805b5d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:06.641251 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:06.644410 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-e33d6dbe-5124-43af-8e10-1ed9805b5d29 None None] MFA Rules not processed for user `fc0ea93084b0482fb2dd6d802987942f`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:01:06.646826 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-e33d6dbe-5124-43af-8e10-1ed9805b5d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:06.760271 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 710/1411] 10.222.0.33 () {58 vars in 1015 bytes} [Sun Sep 11 15:01:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:01:09.333515 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76014b1-fce1-496b-b329-3ffcbea9a2e9 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:09.334099 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76014b1-fce1-496b-b329-3ffcbea9a2e9 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:09.334099 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76014b1-fce1-496b-b329-3ffcbea9a2e9 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:09.335422 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 702/1412] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 15:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:01:09.340600 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed9ad1-4386-4389-9895-f26a9e8798d5 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:09.341004 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed9ad1-4386-4389-9895-f26a9e8798d5 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:09.341143 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed9ad1-4386-4389-9895-f26a9e8798d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:09.408337 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-faed9ad1-4386-4389-9895-f26a9e8798d5 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:01:09.409607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-faed9ad1-4386-4389-9895-f26a9e8798d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:09.465675 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:09.508635 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 711/1413] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 15:01:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:01:09.799633 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76014b1-fce1-496b-b329-3ffcbea9a2e9 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:09.801818 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-c76014b1-fce1-496b-b329-3ffcbea9a2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:09.946707 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76014b1-fce1-496b-b329-3ffcbea9a2e9 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:10.021616 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:10.063785 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cdb7a1-b287-4bf5-bb55-746438dc1f8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc0ea93084b0482fb2dd6d802987942f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:10.064327 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cdb7a1-b287-4bf5-bb55-746438dc1f8e service nova] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:10.064607 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cdb7a1-b287-4bf5-bb55-746438dc1f8e service nova] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:10.064879 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cdb7a1-b287-4bf5-bb55-746438dc1f8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:10.066469 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-23cdb7a1-b287-4bf5-bb55-746438dc1f8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:10.156357 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cdb7a1-b287-4bf5-bb55-746438dc1f8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:10.157967 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cdb7a1-b287-4bf5-bb55-746438dc1f8e service nova] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:10.176215 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 703/1414] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:01:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:01:11.323596 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a792fd-3333-4882-84f7-b64d89f8bef4 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:11.324174 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a792fd-3333-4882-84f7-b64d89f8bef4 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:11.324174 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a792fd-3333-4882-84f7-b64d89f8bef4 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:11.325525 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 712/1415] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 15:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:01:11.331728 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8367d4d-f40f-42f9-88ad-7de58ecb1e13 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:11.332155 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8367d4d-f40f-42f9-88ad-7de58ecb1e13 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:11.332320 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8367d4d-f40f-42f9-88ad-7de58ecb1e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:11.407656 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-f8367d4d-f40f-42f9-88ad-7de58ecb1e13 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:01:11.408873 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-f8367d4d-f40f-42f9-88ad-7de58ecb1e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:11.436193 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:11.507557 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 704/1416] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 15:01:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:01:11.659337 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a792fd-3333-4882-84f7-b64d89f8bef4 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:11.660824 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-28a792fd-3333-4882-84f7-b64d89f8bef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:11.805778 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a792fd-3333-4882-84f7-b64d89f8bef4 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:11.848265 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:11.921835 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef7b6cb-581c-4393-98f8-029ae7821cb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc0ea93084b0482fb2dd6d802987942f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:11.922374 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7b6cb-581c-4393-98f8-029ae7821cb3 service nova] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:11.922652 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7b6cb-581c-4393-98f8-029ae7821cb3 service nova] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:11.922959 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7b6cb-581c-4393-98f8-029ae7821cb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:11.924563 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-eef7b6cb-581c-4393-98f8-029ae7821cb3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:12.015169 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef7b6cb-581c-4393-98f8-029ae7821cb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:12.016798 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef7b6cb-581c-4393-98f8-029ae7821cb3 service nova] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:12.034625 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 713/1417] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:01:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:01:13.172576 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470bc25-84d9-4060-b78c-2d6727b4f9a8 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:13.173194 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470bc25-84d9-4060-b78c-2d6727b4f9a8 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:13.173194 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470bc25-84d9-4060-b78c-2d6727b4f9a8 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:13.174527 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 705/1418] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 15:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:01:13.181004 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6090be-e7e7-4c3f-b43c-70df0393c3d0 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:13.181489 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6090be-e7e7-4c3f-b43c-70df0393c3d0 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:13.181601 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6090be-e7e7-4c3f-b43c-70df0393c3d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:13.235766 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:13.258391 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-4e6090be-e7e7-4c3f-b43c-70df0393c3d0 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:01:13.259631 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-4e6090be-e7e7-4c3f-b43c-70df0393c3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:13.355707 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 714/1419] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 15:01:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:01:13.505724 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a470bc25-84d9-4060-b78c-2d6727b4f9a8 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:13.507183 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-a470bc25-84d9-4060-b78c-2d6727b4f9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:13.653089 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a470bc25-84d9-4060-b78c-2d6727b4f9a8 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:13.665153 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:13.768520 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5a084-bb45-445e-8eab-336e0dc11ade service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc0ea93084b0482fb2dd6d802987942f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:13.769028 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5a084-bb45-445e-8eab-336e0dc11ade service nova] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:13.769306 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5a084-bb45-445e-8eab-336e0dc11ade service nova] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:13.769611 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5a084-bb45-445e-8eab-336e0dc11ade service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:13.771259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-80f5a084-bb45-445e-8eab-336e0dc11ade service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:13.859388 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f5a084-bb45-445e-8eab-336e0dc11ade service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:13.861014 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f5a084-bb45-445e-8eab-336e0dc11ade service nova] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:13.880021 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 706/1420] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:01:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:01:15.025447 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba891b-608c-4a11-a84c-ae191244db0b None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:15.026063 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba891b-608c-4a11-a84c-ae191244db0b None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:15.026407 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba891b-608c-4a11-a84c-ae191244db0b None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:15.028068 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 715/1421] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 15:01:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:01:15.034889 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c90fe-e79a-4804-b0cf-cb689236ddba None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:15.035979 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c90fe-e79a-4804-b0cf-cb689236ddba None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:15.035979 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c90fe-e79a-4804-b0cf-cb689236ddba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:15.057111 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:15.119061 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-dc9c90fe-e79a-4804-b0cf-cb689236ddba None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:01:15.120445 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-dc9c90fe-e79a-4804-b0cf-cb689236ddba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:15.230202 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 707/1422] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 15:01:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:01:15.391503 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba891b-608c-4a11-a84c-ae191244db0b None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:15.393049 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-3aba891b-608c-4a11-a84c-ae191244db0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:15.515566 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:15.557079 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba891b-608c-4a11-a84c-ae191244db0b None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:15.678046 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34bfa51-79c2-43a9-a1e0-b0106ee711e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc0ea93084b0482fb2dd6d802987942f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:15.678691 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bfa51-79c2-43a9-a1e0-b0106ee711e5 service nova] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:15.678988 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bfa51-79c2-43a9-a1e0-b0106ee711e5 service nova] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:15.679261 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bfa51-79c2-43a9-a1e0-b0106ee711e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:15.680850 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-d34bfa51-79c2-43a9-a1e0-b0106ee711e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:15.757833 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:15.773080 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34bfa51-79c2-43a9-a1e0-b0106ee711e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:15.774643 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34bfa51-79c2-43a9-a1e0-b0106ee711e5 service nova] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:15.793432 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 716/1423] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:01:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:01:16.919919 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac2769-a366-4987-969d-91c3e687742e None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:16.920548 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac2769-a366-4987-969d-91c3e687742e None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:16.920548 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac2769-a366-4987-969d-91c3e687742e None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:16.921821 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 708/1424] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 15:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:01:16.928635 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b656f-81f3-48b3-943d-c86cca438659 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:16.929112 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b656f-81f3-48b3-943d-c86cca438659 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:16.929264 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b656f-81f3-48b3-943d-c86cca438659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:17.006741 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-a16b656f-81f3-48b3-943d-c86cca438659 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:01:17.007980 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a16b656f-81f3-48b3-943d-c86cca438659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:17.104273 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 717/1425] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 15:01:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:01:17.257082 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac2769-a366-4987-969d-91c3e687742e None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:17.258519 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-9aac2769-a366-4987-969d-91c3e687742e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:17.341954 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:17.409056 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac2769-a366-4987-969d-91c3e687742e None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:17.522250 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8874e8-9c27-4753-aa86-0682fbb9a5bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc0ea93084b0482fb2dd6d802987942f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:17.522782 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8874e8-9c27-4753-aa86-0682fbb9a5bc service nova] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:17.523089 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8874e8-9c27-4753-aa86-0682fbb9a5bc service nova] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:17.523361 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8874e8-9c27-4753-aa86-0682fbb9a5bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:17.524947 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-ee8874e8-9c27-4753-aa86-0682fbb9a5bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:17.574093 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:17.616569 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8874e8-9c27-4753-aa86-0682fbb9a5bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:17.618195 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8874e8-9c27-4753-aa86-0682fbb9a5bc service nova] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:17.636707 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 709/1426] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:01:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:01:18.695190 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f153440-f735-411f-96b8-625a0f956917 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:18.695769 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f153440-f735-411f-96b8-625a0f956917 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:18.695769 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f153440-f735-411f-96b8-625a0f956917 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:18.697167 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 718/1427] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 15:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:01:18.703600 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a472b-3d8d-4814-9430-876e6601c129 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:18.704046 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a472b-3d8d-4814-9430-876e6601c129 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:18.704163 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a472b-3d8d-4814-9430-876e6601c129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:18.779699 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-a35a472b-3d8d-4814-9430-876e6601c129 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:01:18.780908 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-a35a472b-3d8d-4814-9430-876e6601c129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:18.877682 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 710/1428] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 15:01:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:01:19.028568 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f153440-f735-411f-96b8-625a0f956917 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:19.030096 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7f153440-f735-411f-96b8-625a0f956917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:19.078445 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:19.175326 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f153440-f735-411f-96b8-625a0f956917 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:19.294059 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218299b9-8503-4022-a492-3d550ad7e13a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc0ea93084b0482fb2dd6d802987942f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:19.294682 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218299b9-8503-4022-a492-3d550ad7e13a service nova] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:19.294982 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218299b9-8503-4022-a492-3d550ad7e13a service nova] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:19.295253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218299b9-8503-4022-a492-3d550ad7e13a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:19.296846 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-218299b9-8503-4022-a492-3d550ad7e13a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:19.303989 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:19.392577 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218299b9-8503-4022-a492-3d550ad7e13a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:19.396273 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218299b9-8503-4022-a492-3d550ad7e13a service nova] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:19.415323 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 719/1429] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:01:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:01:20.477880 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8880c-8744-4898-9b42-8e001e4af77d None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:20.478461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8880c-8744-4898-9b42-8e001e4af77d None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:20.478461 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8880c-8744-4898-9b42-8e001e4af77d None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:20.479831 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 711/1430] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 15:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:01:20.485881 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ffe5f1-cba8-4233-b345-01617f64cdb3 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:20.486311 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ffe5f1-cba8-4233-b345-01617f64cdb3 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:20.486431 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ffe5f1-cba8-4233-b345-01617f64cdb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:20.564609 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-35ffe5f1-cba8-4233-b345-01617f64cdb3 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:01:20.565831 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-35ffe5f1-cba8-4233-b345-01617f64cdb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:20.635383 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:20.664060 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 720/1431] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 15:01:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:01:20.815230 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e8880c-8744-4898-9b42-8e001e4af77d None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:20.817237 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-e4e8880c-8744-4898-9b42-8e001e4af77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:20.835012 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:20.963236 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e8880c-8744-4898-9b42-8e001e4af77d None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:21.050374 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:21.077606 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c2913b-eaaa-419d-a624-0a5bc53803c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc0ea93084b0482fb2dd6d802987942f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:21.078211 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2913b-eaaa-419d-a624-0a5bc53803c3 service nova] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:21.078485 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2913b-eaaa-419d-a624-0a5bc53803c3 service nova] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:21.078753 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2913b-eaaa-419d-a624-0a5bc53803c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:21.080393 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-95c2913b-eaaa-419d-a624-0a5bc53803c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:21.168939 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c2913b-eaaa-419d-a624-0a5bc53803c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:21.170553 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c2913b-eaaa-419d-a624-0a5bc53803c3 service nova] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:21.188524 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 712/1432] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:01:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:01:22.251380 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89b931-c887-4877-bf91-d33cbc177b98 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:22.251945 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89b931-c887-4877-bf91-d33cbc177b98 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:22.251945 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89b931-c887-4877-bf91-d33cbc177b98 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:22.253281 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 721/1433] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 15:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:01:22.259487 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1994413c-c509-45e2-a73d-39b75c738354 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:22.259952 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1994413c-c509-45e2-a73d-39b75c738354 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:22.260165 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1994413c-c509-45e2-a73d-39b75c738354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:22.336663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-1994413c-c509-45e2-a73d-39b75c738354 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:01:22.337880 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-1994413c-c509-45e2-a73d-39b75c738354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:22.379605 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:22.434179 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 713/1434] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 15:01:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:01:22.587603 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd89b931-c887-4877-bf91-d33cbc177b98 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:22.589053 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-cd89b931-c887-4877-bf91-d33cbc177b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:22.735784 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd89b931-c887-4877-bf91-d33cbc177b98 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:22.790346 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:22.849071 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009a255-bdfa-4976-a6fb-2e3a76deb112 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc0ea93084b0482fb2dd6d802987942f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:22.849597 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009a255-bdfa-4976-a6fb-2e3a76deb112 service nova] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:22.849875 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009a255-bdfa-4976-a6fb-2e3a76deb112 service nova] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:22.850145 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009a255-bdfa-4976-a6fb-2e3a76deb112 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:22.851763 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-f009a255-bdfa-4976-a6fb-2e3a76deb112 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:22.940192 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f009a255-bdfa-4976-a6fb-2e3a76deb112 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:22.942339 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f009a255-bdfa-4976-a6fb-2e3a76deb112 service nova] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:22.960844 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 722/1435] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:01:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:01:24.018973 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d15b0f-52c4-4995-95ae-b4b64f74132e None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:24.019458 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d15b0f-52c4-4995-95ae-b4b64f74132e None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:24.019567 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d15b0f-52c4-4995-95ae-b4b64f74132e None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:24.020925 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 714/1436] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 15:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:01:24.027300 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e44864-93fb-436b-aa43-86e500a7f453 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:24.027771 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e44864-93fb-436b-aa43-86e500a7f453 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:24.027882 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e44864-93fb-436b-aa43-86e500a7f453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:24.105988 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-62e44864-93fb-436b-aa43-86e500a7f453 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:01:24.107236 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-62e44864-93fb-436b-aa43-86e500a7f453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:24.109253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:24.322440 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 723/1437] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 15:01:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:01:24.474074 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d15b0f-52c4-4995-95ae-b4b64f74132e None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:24.475633 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-43d15b0f-52c4-4995-95ae-b4b64f74132e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:24.622099 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d15b0f-52c4-4995-95ae-b4b64f74132e None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:24.648818 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:24.735435 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd03194-1a98-4a00-b05e-2faab68df9f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc0ea93084b0482fb2dd6d802987942f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:24.736077 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd03194-1a98-4a00-b05e-2faab68df9f8 service nova] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:24.736355 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd03194-1a98-4a00-b05e-2faab68df9f8 service nova] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:24.736627 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd03194-1a98-4a00-b05e-2faab68df9f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:24.739105 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-bbd03194-1a98-4a00-b05e-2faab68df9f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:24.829492 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd03194-1a98-4a00-b05e-2faab68df9f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:24.831148 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd03194-1a98-4a00-b05e-2faab68df9f8 service nova] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:24.849863 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 715/1438] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:01:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:01:26.900568 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297ddb2-8548-465d-b472-3eada2c6d79a None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:26.901124 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297ddb2-8548-465d-b472-3eada2c6d79a None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:26.901124 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297ddb2-8548-465d-b472-3eada2c6d79a None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:26.902454 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 724/1439] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 15:01:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:01:26.907687 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f61b6d-a06b-4424-9e9f-aca15ba521ce None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:26.908115 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f61b6d-a06b-4424-9e9f-aca15ba521ce None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:26.908212 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f61b6d-a06b-4424-9e9f-aca15ba521ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:26.951620 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:26.977961 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-29f61b6d-a06b-4424-9e9f-aca15ba521ce None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:01:26.979234 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-29f61b6d-a06b-4424-9e9f-aca15ba521ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:27.076740 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 716/1440] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 15:01:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:01:27.365088 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7297ddb2-8548-465d-b472-3eada2c6d79a None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:27.366617 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7297ddb2-8548-465d-b472-3eada2c6d79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:27.470559 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7297ddb2-8548-465d-b472-3eada2c6d79a None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:27.503207 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:27.544995 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d2c75b-9349-4f5b-adaf-dbf404266a5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6eb236e80b4395a7aa77020b2f3a4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:27.545596 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d2c75b-9349-4f5b-adaf-dbf404266a5b service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:27.545872 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d2c75b-9349-4f5b-adaf-dbf404266a5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:27.546166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d2c75b-9349-4f5b-adaf-dbf404266a5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:27.547773 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-80d2c75b-9349-4f5b-adaf-dbf404266a5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:27.660407 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d2c75b-9349-4f5b-adaf-dbf404266a5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:27.661773 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d2c75b-9349-4f5b-adaf-dbf404266a5b service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:27.679676 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 725/1441] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:01:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:01:29.989708 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d873697-cff6-4106-8c56-2d1db5130718 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:29.990283 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d873697-cff6-4106-8c56-2d1db5130718 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:29.990385 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d873697-cff6-4106-8c56-2d1db5130718 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:29.991817 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 717/1442] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 15:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:01:29.997399 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957e69d-a4ae-4bd7-9d16-b621b3863c1d None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:29.997867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957e69d-a4ae-4bd7-9d16-b621b3863c1d None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:29.997965 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957e69d-a4ae-4bd7-9d16-b621b3863c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:30.065336 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-e957e69d-a4ae-4bd7-9d16-b621b3863c1d None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:01:30.067354 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-e957e69d-a4ae-4bd7-9d16-b621b3863c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:30.079745 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:30.163186 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 726/1443] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 15:01:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:01:30.450520 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d873697-cff6-4106-8c56-2d1db5130718 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:30.452111 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-0d873697-cff6-4106-8c56-2d1db5130718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:30.554305 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d873697-cff6-4106-8c56-2d1db5130718 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:30.567350 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:30.633568 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce2e586-54c2-40a6-912a-836bd1717fb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6eb236e80b4395a7aa77020b2f3a4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:30.634182 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2e586-54c2-40a6-912a-836bd1717fb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:30.634464 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2e586-54c2-40a6-912a-836bd1717fb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:30.634746 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2e586-54c2-40a6-912a-836bd1717fb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:30.636384 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-4ce2e586-54c2-40a6-912a-836bd1717fb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:30.747846 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce2e586-54c2-40a6-912a-836bd1717fb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:30.749326 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce2e586-54c2-40a6-912a-836bd1717fb4 service neutron] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:30.768892 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 718/1444] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:01:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:01:33.220433 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f956ca1-c265-4e05-81b1-4170ecda0f91 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:33.220974 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f956ca1-c265-4e05-81b1-4170ecda0f91 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:33.220974 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f956ca1-c265-4e05-81b1-4170ecda0f91 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:33.223212 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 727/1445] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 15:01:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:01:33.228756 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3f81a-1685-405d-bbfc-52e1d3ea8efd None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:33.229233 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3f81a-1685-405d-bbfc-52e1d3ea8efd None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:33.229329 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3f81a-1685-405d-bbfc-52e1d3ea8efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:33.276192 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:33.296440 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-6bb3f81a-1685-405d-bbfc-52e1d3ea8efd None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:01:33.297687 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-6bb3f81a-1685-405d-bbfc-52e1d3ea8efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:33.393144 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 719/1446] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 15:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:01:33.697530 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867381ee-d024-4a38-92e4-807dc8dab8df None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:33.697811 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867381ee-d024-4a38-92e4-807dc8dab8df None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:33.698086 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867381ee-d024-4a38-92e4-807dc8dab8df None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:33.699507 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 728/1447] 10.222.0.33 () {54 vars in 965 bytes} [Sun Sep 11 15:01:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:01:33.705604 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f1eda-9a47-4c1f-a4c3-f43d2c5d7d8b None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:33.705890 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f1eda-9a47-4c1f-a4c3-f43d2c5d7d8b None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:33.706167 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f1eda-9a47-4c1f-a4c3-f43d2c5d7d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:33.781786 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-008f1eda-9a47-4c1f-a4c3-f43d2c5d7d8b None None] MFA Rules not processed for user `df6eb236e80b4395a7aa77020b2f3a4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:01:33.783031 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-008f1eda-9a47-4c1f-a4c3-f43d2c5d7d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:33.842682 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:33.860844 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 720/1448] 10.222.0.33 () {58 vars in 1062 bytes} [Sun Sep 11 15:01:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:01:33.870584 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867381ee-d024-4a38-92e4-807dc8dab8df None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:33.872143 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-867381ee-d024-4a38-92e4-807dc8dab8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:33.973677 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867381ee-d024-4a38-92e4-807dc8dab8df None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:34.015825 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:34.050563 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e660ac-ec80-494c-be86-6369fa8352ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6eb236e80b4395a7aa77020b2f3a4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:34.051444 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e660ac-ec80-494c-be86-6369fa8352ba service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:34.051715 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e660ac-ec80-494c-be86-6369fa8352ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:34.051977 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e660ac-ec80-494c-be86-6369fa8352ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:34.053562 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-99e660ac-ec80-494c-be86-6369fa8352ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:34.161886 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e660ac-ec80-494c-be86-6369fa8352ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:34.163387 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e660ac-ec80-494c-be86-6369fa8352ba service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:34.181604 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 729/1449] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:01:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:01:36.882156 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ff1f0-2524-4b8c-96ab-4260abfcf702 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:36.882725 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ff1f0-2524-4b8c-96ab-4260abfcf702 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:36.882725 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ff1f0-2524-4b8c-96ab-4260abfcf702 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:36.884234 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 721/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 15:01:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:01:36.889436 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc3a71-538d-484e-8bd3-c5c42d92c270 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:36.889872 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc3a71-538d-484e-8bd3-c5c42d92c270 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:36.889979 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc3a71-538d-484e-8bd3-c5c42d92c270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:36.955743 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-e7fc3a71-538d-484e-8bd3-c5c42d92c270 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:01:36.957490 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-e7fc3a71-538d-484e-8bd3-c5c42d92c270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:36.978618 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:37.056228 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 730/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 15:01:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:01:37.340118 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ff1f0-2524-4b8c-96ab-4260abfcf702 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:37.341669 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-e35ff1f0-2524-4b8c-96ab-4260abfcf702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:37.445910 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ff1f0-2524-4b8c-96ab-4260abfcf702 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:37.519268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921e6cb-756a-493a-bc52-0afd63a8ea0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6eb236e80b4395a7aa77020b2f3a4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:37.519875 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921e6cb-756a-493a-bc52-0afd63a8ea0f service neutron] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:37.520160 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921e6cb-756a-493a-bc52-0afd63a8ea0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:37.520429 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921e6cb-756a-493a-bc52-0afd63a8ea0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:37.522037 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-a921e6cb-756a-493a-bc52-0afd63a8ea0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:37.563922 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:37.634302 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a921e6cb-756a-493a-bc52-0afd63a8ea0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:37.635729 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a921e6cb-756a-493a-bc52-0afd63a8ea0f service neutron] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:37.654493 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 722/1452] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:01:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:01:39.863940 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d957252-a3e4-42e0-8d64-ec95f1bdb8c6 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:39.864502 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d957252-a3e4-42e0-8d64-ec95f1bdb8c6 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:39.864502 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d957252-a3e4-42e0-8d64-ec95f1bdb8c6 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:39.865842 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 731/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 15:01:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:01:39.871247 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142aa95-4e10-4db9-8c22-cbb3cbb0d631 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:39.871653 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142aa95-4e10-4db9-8c22-cbb3cbb0d631 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:39.871789 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142aa95-4e10-4db9-8c22-cbb3cbb0d631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:39.936986 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-9142aa95-4e10-4db9-8c22-cbb3cbb0d631 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:01:39.938446 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-9142aa95-4e10-4db9-8c22-cbb3cbb0d631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:40.030485 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:40.036976 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 723/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 15:01:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:01:40.326292 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d957252-a3e4-42e0-8d64-ec95f1bdb8c6 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:40.327849 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-7d957252-a3e4-42e0-8d64-ec95f1bdb8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:40.429781 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d957252-a3e4-42e0-8d64-ec95f1bdb8c6 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:40.478608 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:40.503381 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d82697-83a9-4577-9491-7b15ce8b656b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6eb236e80b4395a7aa77020b2f3a4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:40.503976 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d82697-83a9-4577-9491-7b15ce8b656b service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:40.504253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d82697-83a9-4577-9491-7b15ce8b656b service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:40.504521 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d82697-83a9-4577-9491-7b15ce8b656b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:40.506092 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-33d82697-83a9-4577-9491-7b15ce8b656b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:40.614166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d82697-83a9-4577-9491-7b15ce8b656b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:40.615548 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d82697-83a9-4577-9491-7b15ce8b656b service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:40.634138 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 732/1455] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:01:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:01:42.898102 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a2d32-54ab-4d4d-92dc-67b7ed726a76 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:42.898766 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a2d32-54ab-4d4d-92dc-67b7ed726a76 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:42.898766 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a2d32-54ab-4d4d-92dc-67b7ed726a76 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:42.900028 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 724/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 15:01:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:01:42.905316 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f934f-20bb-44b4-9599-b0ca56d06a74 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:42.905743 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f934f-20bb-44b4-9599-b0ca56d06a74 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:42.905847 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f934f-20bb-44b4-9599-b0ca56d06a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:42.973429 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-a26f934f-20bb-44b4-9599-b0ca56d06a74 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:01:42.974644 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a26f934f-20bb-44b4-9599-b0ca56d06a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:43.004469 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:43.073702 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 733/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 15:01:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:01:43.358617 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a2d32-54ab-4d4d-92dc-67b7ed726a76 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:43.360220 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-f84a2d32-54ab-4d4d-92dc-67b7ed726a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:43.463225 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a2d32-54ab-4d4d-92dc-67b7ed726a76 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:43.537604 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1fef04-e031-4d40-8df4-4e28c08eeab9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6eb236e80b4395a7aa77020b2f3a4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:43.538209 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1fef04-e031-4d40-8df4-4e28c08eeab9 service neutron] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:43.538489 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1fef04-e031-4d40-8df4-4e28c08eeab9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:43.538768 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1fef04-e031-4d40-8df4-4e28c08eeab9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:43.540421 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-bd1fef04-e031-4d40-8df4-4e28c08eeab9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:43.707052 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:43.767364 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1fef04-e031-4d40-8df4-4e28c08eeab9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:43.769672 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1fef04-e031-4d40-8df4-4e28c08eeab9 service neutron] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:43.788180 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 725/1458] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:01:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:01:46.006391 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5e4cc-3509-418b-9bb7-565e4b0137b9 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:46.006891 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5e4cc-3509-418b-9bb7-565e4b0137b9 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:46.006992 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5e4cc-3509-418b-9bb7-565e4b0137b9 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:46.008334 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 734/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 15:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:01:46.013798 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd9d86-1781-44fc-86c0-1b5ed4d45e05 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:46.014208 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd9d86-1781-44fc-86c0-1b5ed4d45e05 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:46.014321 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd9d86-1781-44fc-86c0-1b5ed4d45e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:46.079689 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-e9fd9d86-1781-44fc-86c0-1b5ed4d45e05 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:01:46.081446 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-e9fd9d86-1781-44fc-86c0-1b5ed4d45e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:46.176095 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 726/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 15:01:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:01:46.461592 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5e4cc-3509-418b-9bb7-565e4b0137b9 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:46.463192 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-fbf5e4cc-3509-418b-9bb7-565e4b0137b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:46.566821 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5e4cc-3509-418b-9bb7-565e4b0137b9 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:46.623471 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:46.641639 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b669560c-f5f0-4be7-83a9-5e75af6f1b98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6eb236e80b4395a7aa77020b2f3a4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:46.642253 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669560c-f5f0-4be7-83a9-5e75af6f1b98 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:46.642530 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669560c-f5f0-4be7-83a9-5e75af6f1b98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:46.642813 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669560c-f5f0-4be7-83a9-5e75af6f1b98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:46.645049 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-b669560c-f5f0-4be7-83a9-5e75af6f1b98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:46.756050 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b669560c-f5f0-4be7-83a9-5e75af6f1b98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:46.757525 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b669560c-f5f0-4be7-83a9-5e75af6f1b98 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:46.776273 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 735/1461] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:01:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:01:48.977412 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2254804-5043-402f-bd2e-3c40c8c610f9 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:48.978001 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2254804-5043-402f-bd2e-3c40c8c610f9 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:48.978001 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2254804-5043-402f-bd2e-3c40c8c610f9 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:48.979321 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 727/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 15:01:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:01:48.984657 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a207ae-798a-41b4-9c80-ec8e7f5d79a2 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:48.985086 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a207ae-798a-41b4-9c80-ec8e7f5d79a2 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:48.985204 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a207ae-798a-41b4-9c80-ec8e7f5d79a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:49.051599 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-f2a207ae-798a-41b4-9c80-ec8e7f5d79a2 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:01:49.052801 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-f2a207ae-798a-41b4-9c80-ec8e7f5d79a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:49.090975 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:49.148977 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 736/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 15:01:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:01:49.431181 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2254804-5043-402f-bd2e-3c40c8c610f9 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:49.432761 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-a2254804-5043-402f-bd2e-3c40c8c610f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:49.436590 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:49.536223 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2254804-5043-402f-bd2e-3c40c8c610f9 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:49.616926 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd7b26-d9db-482a-bec2-edf3bdefc157 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6eb236e80b4395a7aa77020b2f3a4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:49.617455 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd7b26-d9db-482a-bec2-edf3bdefc157 service neutron] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:49.617770 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd7b26-d9db-482a-bec2-edf3bdefc157 service neutron] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:49.618041 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd7b26-d9db-482a-bec2-edf3bdefc157 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:49.619680 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-2fdd7b26-d9db-482a-bec2-edf3bdefc157 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:49.679388 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:49.729624 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdd7b26-d9db-482a-bec2-edf3bdefc157 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:49.731113 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdd7b26-d9db-482a-bec2-edf3bdefc157 service neutron] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:49.749483 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 728/1464] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:01:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:01:51.915665 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf9e6d-8725-43e7-b75f-b4b73812b389 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:51.916256 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf9e6d-8725-43e7-b75f-b4b73812b389 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:51.916256 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf9e6d-8725-43e7-b75f-b4b73812b389 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:51.917560 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 737/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 15:01:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:01:51.922771 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61301d0-a86f-4e85-b090-a7d50edc29be None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:51.923243 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61301d0-a86f-4e85-b090-a7d50edc29be None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:51.923360 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61301d0-a86f-4e85-b090-a7d50edc29be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:51.989569 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-b61301d0-a86f-4e85-b090-a7d50edc29be None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:01:51.990775 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-b61301d0-a86f-4e85-b090-a7d50edc29be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:52.087676 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 729/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 15:01:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:01:52.371615 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdf9e6d-8725-43e7-b75f-b4b73812b389 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:52.373144 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-8cdf9e6d-8725-43e7-b75f-b4b73812b389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:52.476309 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdf9e6d-8725-43e7-b75f-b4b73812b389 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:52.542287 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:52.550108 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4998867-5575-462f-af2f-14df175b77de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6eb236e80b4395a7aa77020b2f3a4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:52.550934 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4998867-5575-462f-af2f-14df175b77de service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:52.551236 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4998867-5575-462f-af2f-14df175b77de service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:52.551499 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4998867-5575-462f-af2f-14df175b77de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:52.553106 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a4998867-5575-462f-af2f-14df175b77de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:52.663624 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4998867-5575-462f-af2f-14df175b77de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:52.664991 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4998867-5575-462f-af2f-14df175b77de service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:52.682871 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 738/1467] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:01:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:01:54.849245 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a484f-2ca1-42f4-b76e-104a3c57d51d None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:54.849839 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a484f-2ca1-42f4-b76e-104a3c57d51d None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:54.849839 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a484f-2ca1-42f4-b76e-104a3c57d51d None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:54.851160 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 730/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 15:01:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:01:54.856452 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d74176-2fbd-4df2-9a5b-7a8aabad7cd0 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:54.856867 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d74176-2fbd-4df2-9a5b-7a8aabad7cd0 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:54.856983 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d74176-2fbd-4df2-9a5b-7a8aabad7cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:54.921911 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-98d74176-2fbd-4df2-9a5b-7a8aabad7cd0 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:01:54.923216 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-98d74176-2fbd-4df2-9a5b-7a8aabad7cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:54.971779 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:55.021350 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 739/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 15:01:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:01:55.304716 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a484f-2ca1-42f4-b76e-104a3c57d51d None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:55.306298 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-3d0a484f-2ca1-42f4-b76e-104a3c57d51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:55.320066 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:55.409120 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a484f-2ca1-42f4-b76e-104a3c57d51d None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:55.480937 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d188f6c-8ee1-4e60-919c-4bcbab44b3ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6eb236e80b4395a7aa77020b2f3a4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:55.481539 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d188f6c-8ee1-4e60-919c-4bcbab44b3ca service neutron] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:55.481818 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d188f6c-8ee1-4e60-919c-4bcbab44b3ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:55.482086 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d188f6c-8ee1-4e60-919c-4bcbab44b3ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:55.483704 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-2d188f6c-8ee1-4e60-919c-4bcbab44b3ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:55.551692 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:55.593155 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d188f6c-8ee1-4e60-919c-4bcbab44b3ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:55.594588 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d188f6c-8ee1-4e60-919c-4bcbab44b3ca service neutron] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:55.613872 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 731/1470] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:01:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:02:05.019794 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=22 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:05.611422 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=25 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:15.620935 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4f501-5da1-4e96-92f1-be7752af2245 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:15.621554 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4f501-5da1-4e96-92f1-be7752af2245 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:15.621554 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4f501-5da1-4e96-92f1-be7752af2245 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:15.622886 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 740/1471] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 15:02:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:02:15.628324 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a260b-dedc-47f1-b2bb-963f599f187e None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:15.628774 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a260b-dedc-47f1-b2bb-963f599f187e None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:15.628902 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a260b-dedc-47f1-b2bb-963f599f187e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:15.694604 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-1f3a260b-dedc-47f1-b2bb-963f599f187e None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:02:15.695841 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-1f3a260b-dedc-47f1-b2bb-963f599f187e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:02:15.793475 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 732/1472] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 15:02:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:02:16.099865 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa2ce8-037f-4a21-9ec8-dbe43c847850 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:16.100151 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa2ce8-037f-4a21-9ec8-dbe43c847850 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:16.100424 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa2ce8-037f-4a21-9ec8-dbe43c847850 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:16.101752 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 741/1473] 10.222.0.33 () {54 vars in 966 bytes} [Sun Sep 11 15:02:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:02:16.106663 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de06352-99a6-4ea0-810c-bec6383f2bfd None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:16.106990 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de06352-99a6-4ea0-810c-bec6383f2bfd None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:16.107262 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de06352-99a6-4ea0-810c-bec6383f2bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:16.182382 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-0de06352-99a6-4ea0-810c-bec6383f2bfd None None] MFA Rules not processed for user `cd9c8b53871e415facea40c6694f4171`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:02:16.183625 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-0de06352-99a6-4ea0-810c-bec6383f2bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:02:16.204533 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:16.261843 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 733/1474] 10.222.0.33 () {58 vars in 1063 bytes} [Sun Sep 11 15:02:16 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:02:16.270167 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa2ce8-037f-4a21-9ec8-dbe43c847850 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:16.271729 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-24aa2ce8-037f-4a21-9ec8-dbe43c847850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:02:16.373866 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa2ce8-037f-4a21-9ec8-dbe43c847850 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:16.447934 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d52853-45b2-4bd1-9b11-ef3b74ef6817 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd9c8b53871e415facea40c6694f4171', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:16.448532 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d52853-45b2-4bd1-9b11-ef3b74ef6817 service glance] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:16.448809 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d52853-45b2-4bd1-9b11-ef3b74ef6817 service glance] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:16.449079 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d52853-45b2-4bd1-9b11-ef3b74ef6817 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:16.450662 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-b7d52853-45b2-4bd1-9b11-ef3b74ef6817 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:02:16.504360 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:16.560058 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d52853-45b2-4bd1-9b11-ef3b74ef6817 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:16.561428 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d52853-45b2-4bd1-9b11-ef3b74ef6817 service glance] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:16.581851 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 742/1475] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:02:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:02:18.851667 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375f303-dcc2-49d6-ae61-de6acb3e6b38 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:18.852265 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375f303-dcc2-49d6-ae61-de6acb3e6b38 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:18.852265 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375f303-dcc2-49d6-ae61-de6acb3e6b38 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:18.853570 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 734/1476] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 15:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:02:18.858936 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a2f7dc-3539-4d40-a98a-c3fce7e17ef3 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:18.859383 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a2f7dc-3539-4d40-a98a-c3fce7e17ef3 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:18.859505 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a2f7dc-3539-4d40-a98a-c3fce7e17ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:18.925955 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-29a2f7dc-3539-4d40-a98a-c3fce7e17ef3 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:02:18.927200 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-29a2f7dc-3539-4d40-a98a-c3fce7e17ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:02:19.023023 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 743/1477] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 15:02:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:02:19.301026 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375f303-dcc2-49d6-ae61-de6acb3e6b38 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:19.302553 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-1375f303-dcc2-49d6-ae61-de6acb3e6b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:02:19.429599 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375f303-dcc2-49d6-ae61-de6acb3e6b38 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:19.478249 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:19.524887 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf09c69-9c99-438c-aacb-63208b62755f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b748811b1c8a461188c96c2e314bf507', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:19.525483 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf09c69-9c99-438c-aacb-63208b62755f admin admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:19.525757 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf09c69-9c99-438c-aacb-63208b62755f admin admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:19.526030 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf09c69-9c99-438c-aacb-63208b62755f admin admin] PATH_INFO: `/v3/projects` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:19.526651 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf09c69-9c99-438c-aacb-63208b62755f admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:19.528411 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: WARNING py.warnings [None req-edf09c69-9c99-438c-aacb-63208b62755f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 15:02:19.528411 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: warnings.warn(msg) Sep 11 15:02:19.528411 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: Sep 11 15:02:19.529551 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf09c69-9c99-438c-aacb-63208b62755f admin admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:19.550217 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 735/1478] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 15:02:19 2022] GET /identity/v3/projects => generated 2553 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:02:21.726143 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe9aa0-3c87-43ff-8a0d-7e01f4963789 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:21.726740 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe9aa0-3c87-43ff-8a0d-7e01f4963789 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:21.726740 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe9aa0-3c87-43ff-8a0d-7e01f4963789 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:21.728061 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 744/1479] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 15:02:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:02:21.733428 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3530fe-ce97-4f8f-adb5-c51dc61bd088 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:21.733858 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3530fe-ce97-4f8f-adb5-c51dc61bd088 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:21.733975 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3530fe-ce97-4f8f-adb5-c51dc61bd088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:21.806088 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-3c3530fe-ce97-4f8f-adb5-c51dc61bd088 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:02:21.807328 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-3c3530fe-ce97-4f8f-adb5-c51dc61bd088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:02:21.906340 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 736/1480] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 15:02:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:02:22.200151 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe9aa0-3c87-43ff-8a0d-7e01f4963789 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:22.201799 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-0fbe9aa0-3c87-43ff-8a0d-7e01f4963789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:02:22.211792 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:22.350192 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe9aa0-3c87-43ff-8a0d-7e01f4963789 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:22.430589 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:22.463134 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29711dd-7188-4060-98f4-00e520add3f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc0ea93084b0482fb2dd6d802987942f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:22.463742 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29711dd-7188-4060-98f4-00e520add3f0 service nova] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:22.464014 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29711dd-7188-4060-98f4-00e520add3f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:22.464279 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29711dd-7188-4060-98f4-00e520add3f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:22.465853 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-f29711dd-7188-4060-98f4-00e520add3f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:02:22.555320 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29711dd-7188-4060-98f4-00e520add3f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:22.556917 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29711dd-7188-4060-98f4-00e520add3f0 service nova] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:22.574966 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 745/1481] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:02:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:02:24.907912 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a63dd4-0aca-41c7-95d9-2ce1cee5f4b4 None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:24.908490 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a63dd4-0aca-41c7-95d9-2ce1cee5f4b4 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:24.908490 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a63dd4-0aca-41c7-95d9-2ce1cee5f4b4 None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:24.909837 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 737/1482] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 15:02:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:02:24.915363 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205cc924-5f2f-431d-ae0e-6c5f8dc58e95 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:24.915796 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205cc924-5f2f-431d-ae0e-6c5f8dc58e95 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:24.915907 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205cc924-5f2f-431d-ae0e-6c5f8dc58e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:24.982135 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-205cc924-5f2f-431d-ae0e-6c5f8dc58e95 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:02:24.983396 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-205cc924-5f2f-431d-ae0e-6c5f8dc58e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:02:25.030682 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:25.082361 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 746/1483] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 15:02:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:02:25.370459 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a63dd4-0aca-41c7-95d9-2ce1cee5f4b4 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:25.372090 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-49a63dd4-0aca-41c7-95d9-2ce1cee5f4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:02:25.396571 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:25.476104 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a63dd4-0aca-41c7-95d9-2ce1cee5f4b4 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:25.550624 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9bed03-62ec-42c1-b58d-4ec50840b6e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6eb236e80b4395a7aa77020b2f3a4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:25.551261 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9bed03-62ec-42c1-b58d-4ec50840b6e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:25.551543 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9bed03-62ec-42c1-b58d-4ec50840b6e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:25.551818 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9bed03-62ec-42c1-b58d-4ec50840b6e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:25.553413 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-8f9bed03-62ec-42c1-b58d-4ec50840b6e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:02:25.626906 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:25.662497 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9bed03-62ec-42c1-b58d-4ec50840b6e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:25.663981 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9bed03-62ec-42c1-b58d-4ec50840b6e2 service neutron] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:25.682296 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 738/1484] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:02:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:02:27.888884 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0cded4-368d-437d-af5c-aa47e5269e74 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:27.889441 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0cded4-368d-437d-af5c-aa47e5269e74 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:27.889441 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0cded4-368d-437d-af5c-aa47e5269e74 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:27.890885 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 747/1485] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 15:02:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:02:27.896886 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e2f14-ff47-4e5d-bb52-797dd58e72a0 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:27.897302 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e2f14-ff47-4e5d-bb52-797dd58e72a0 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:27.897432 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e2f14-ff47-4e5d-bb52-797dd58e72a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:27.962779 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-df8e2f14-ff47-4e5d-bb52-797dd58e72a0 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:02:27.964022 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-df8e2f14-ff47-4e5d-bb52-797dd58e72a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:02:28.060757 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 739/1486] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 15:02:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:02:28.350713 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0cded4-368d-437d-af5c-aa47e5269e74 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:28.352275 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-5c0cded4-368d-437d-af5c-aa47e5269e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:02:28.456227 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0cded4-368d-437d-af5c-aa47e5269e74 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:28.529090 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:28.530735 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b6b806-f429-42cb-a16a-2e96c2f054fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6eb236e80b4395a7aa77020b2f3a4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:28.531436 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b6b806-f429-42cb-a16a-2e96c2f054fb service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:28.531900 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b6b806-f429-42cb-a16a-2e96c2f054fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:28.532930 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b6b806-f429-42cb-a16a-2e96c2f054fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:28.534741 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-75b6b806-f429-42cb-a16a-2e96c2f054fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:02:28.643922 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b6b806-f429-42cb-a16a-2e96c2f054fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:28.645291 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b6b806-f429-42cb-a16a-2e96c2f054fb service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:28.663472 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 748/1487] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:02:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:02:30.969029 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82301475-af07-47c6-9c39-c25fe0cc1c5d None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:30.969585 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82301475-af07-47c6-9c39-c25fe0cc1c5d None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:30.969585 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82301475-af07-47c6-9c39-c25fe0cc1c5d None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:30.970970 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 740/1488] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 15:02:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:02:30.976202 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa144753-ac5f-4907-b136-2100ab5aa9c3 None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:30.976650 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa144753-ac5f-4907-b136-2100ab5aa9c3 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:30.976764 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa144753-ac5f-4907-b136-2100ab5aa9c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:31.043858 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-aa144753-ac5f-4907-b136-2100ab5aa9c3 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:02:31.045061 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-aa144753-ac5f-4907-b136-2100ab5aa9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:02:31.097659 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:31.141297 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 749/1489] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 15:02:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:02:31.430221 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82301475-af07-47c6-9c39-c25fe0cc1c5d None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:31.431792 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-82301475-af07-47c6-9c39-c25fe0cc1c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:02:31.468450 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:31.536242 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82301475-af07-47c6-9c39-c25fe0cc1c5d None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:31.613152 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89f76b-88e7-4629-9506-daa8c48706a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6eb236e80b4395a7aa77020b2f3a4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:31.613794 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89f76b-88e7-4629-9506-daa8c48706a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:31.614073 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89f76b-88e7-4629-9506-daa8c48706a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:31.614343 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89f76b-88e7-4629-9506-daa8c48706a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:31.616018 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-0b89f76b-88e7-4629-9506-daa8c48706a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:02:31.697428 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:31.726672 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89f76b-88e7-4629-9506-daa8c48706a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:31.728268 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89f76b-88e7-4629-9506-daa8c48706a8 service neutron] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:31.746729 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 741/1490] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:02:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:02:34.097984 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce35f1f-019a-468b-ace0-d3ce46ea2ad5 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:34.098550 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce35f1f-019a-468b-ace0-d3ce46ea2ad5 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:34.098550 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce35f1f-019a-468b-ace0-d3ce46ea2ad5 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:34.099935 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 750/1491] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 15:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:02:34.105282 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491464a-edc4-41c3-aa5e-ef66e2758612 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:34.105694 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491464a-edc4-41c3-aa5e-ef66e2758612 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:34.105804 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491464a-edc4-41c3-aa5e-ef66e2758612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:34.172069 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-9491464a-edc4-41c3-aa5e-ef66e2758612 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:02:34.173334 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-9491464a-edc4-41c3-aa5e-ef66e2758612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:02:34.270194 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 742/1492] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 15:02:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:02:34.565131 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce35f1f-019a-468b-ace0-d3ce46ea2ad5 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:34.566653 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-4ce35f1f-019a-468b-ace0-d3ce46ea2ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:02:34.673372 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce35f1f-019a-468b-ace0-d3ce46ea2ad5 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:34.748672 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8632e4-518c-4fc5-a24c-acaaf7870210 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6eb236e80b4395a7aa77020b2f3a4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:34.749278 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8632e4-518c-4fc5-a24c-acaaf7870210 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:34.749557 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8632e4-518c-4fc5-a24c-acaaf7870210 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:34.749826 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8632e4-518c-4fc5-a24c-acaaf7870210 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:34.751527 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-3a8632e4-518c-4fc5-a24c-acaaf7870210 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:02:34.760032 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:34.860402 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8632e4-518c-4fc5-a24c-acaaf7870210 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:34.861740 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8632e4-518c-4fc5-a24c-acaaf7870210 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:34.880158 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 751/1493] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:02:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:02:44.268621 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=82 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:44.877847 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=50 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:03:47.579445 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57aed0-eaa0-4511-bcbb-a7806fa33601 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:47.580059 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57aed0-eaa0-4511-bcbb-a7806fa33601 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:47.580059 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57aed0-eaa0-4511-bcbb-a7806fa33601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:47.655011 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-4f57aed0-eaa0-4511-bcbb-a7806fa33601 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:03:47.656399 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-4f57aed0-eaa0-4511-bcbb-a7806fa33601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:03:47.752733 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 743/1494] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 15:03:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:03:47.760671 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8aced0-7178-4997-95d4-f3d95aad94ef None None] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:47.761184 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8aced0-7178-4997-95d4-f3d95aad94ef None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:47.761291 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8aced0-7178-4997-95d4-f3d95aad94ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:47.841675 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.auth.core [None req-8b8aced0-7178-4997-95d4-f3d95aad94ef None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81430) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:03:47.843571 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-8b8aced0-7178-4997-95d4-f3d95aad94ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:03:47.938581 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 752/1495] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 15:03:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:03:47.943163 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57aed0-eaa0-4511-bcbb-a7806fa33601 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:03:47.944399 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-4f57aed0-eaa0-4511-bcbb-a7806fa33601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:03:47.955925 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57aed0-eaa0-4511-bcbb-a7806fa33601 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:03:48.018220 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:03:48.052186 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a1b36-f8d1-40f0-8bd6-cd63eb0cdf3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b748811b1c8a461188c96c2e314bf507', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:03:48.052804 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a1b36-f8d1-40f0-8bd6-cd63eb0cdf3d admin admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:48.053084 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a1b36-f8d1-40f0-8bd6-cd63eb0cdf3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:48.053353 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a1b36-f8d1-40f0-8bd6-cd63eb0cdf3d admin admin] PATH_INFO: `/v3/domains` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:48.053992 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a1b36-f8d1-40f0-8bd6-cd63eb0cdf3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81431) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:03:48.054288 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a1b36-f8d1-40f0-8bd6-cd63eb0cdf3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:03:48.055326 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: WARNING py.warnings [None req-2b9a1b36-f8d1-40f0-8bd6-cd63eb0cdf3d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 11 15:03:48.055326 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: warnings.warn(msg) Sep 11 15:03:48.055326 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: Sep 11 15:03:48.056223 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a1b36-f8d1-40f0-8bd6-cd63eb0cdf3d admin admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:03:48.067483 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 744/1496] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 15:03:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:03:48.075715 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8aced0-7178-4997-95d4-f3d95aad94ef None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:03:48.085729 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8aced0-7178-4997-95d4-f3d95aad94ef None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:03:48.147999 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:03:48.184088 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d2040-bbab-4b9e-a4dd-95e4fa1bfc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b748811b1c8a461188c96c2e314bf507', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:03:48.184701 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d2040-bbab-4b9e-a4dd-95e4fa1bfc35 admin admin] REQUEST_METHOD: `POST` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:48.184979 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d2040-bbab-4b9e-a4dd-95e4fa1bfc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:48.185254 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d2040-bbab-4b9e-a4dd-95e4fa1bfc35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:48.185887 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13d2040-bbab-4b9e-a4dd-95e4fa1bfc35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:03:48.187916 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: WARNING py.warnings [None req-a13d2040-bbab-4b9e-a4dd-95e4fa1bfc35 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 15:03:48.187916 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: warnings.warn(msg) Sep 11 15:03:48.187916 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: Sep 11 15:03:48.189146 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13d2040-bbab-4b9e-a4dd-95e4fa1bfc35 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:03:48.247124 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 753/1497] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 15:03:48 2022] POST /identity/v3/projects => generated 375 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:03:48.251344 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a1b36-f8d1-40f0-8bd6-cd63eb0cdf3d admin admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:03:48.253230 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-2b9a1b36-f8d1-40f0-8bd6-cd63eb0cdf3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:03:48.287427 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a1b36-f8d1-40f0-8bd6-cd63eb0cdf3d admin admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:03:48.381929 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc848762-1679-4aaf-84e9-5239fad2bf75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b748811b1c8a461188c96c2e314bf507', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:03:48.382537 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc848762-1679-4aaf-84e9-5239fad2bf75 admin admin] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:48.382829 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc848762-1679-4aaf-84e9-5239fad2bf75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:48.383124 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc848762-1679-4aaf-84e9-5239fad2bf75 admin admin] PATH_INFO: `/v3/users` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:48.383754 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc848762-1679-4aaf-84e9-5239fad2bf75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:03:48.384761 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: WARNING py.warnings [None req-cc848762-1679-4aaf-84e9-5239fad2bf75 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 15:03:48.384761 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: warnings.warn(msg) Sep 11 15:03:48.384761 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: Sep 11 15:03:48.385988 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc848762-1679-4aaf-84e9-5239fad2bf75 admin admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:03:48.428497 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 745/1498] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 15:03:48 2022] POST /identity/v3/users => generated 288 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:03:48.432494 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d2040-bbab-4b9e-a4dd-95e4fa1bfc35 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:03:48.433991 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-a13d2040-bbab-4b9e-a4dd-95e4fa1bfc35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:03:48.495687 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d2040-bbab-4b9e-a4dd-95e4fa1bfc35 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:03:48.580340 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:03:48.591106 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880e37aa-8502-4746-9fd3-170ee114012a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b748811b1c8a461188c96c2e314bf507', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:03:48.591709 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e37aa-8502-4746-9fd3-170ee114012a admin admin] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:48.591981 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e37aa-8502-4746-9fd3-170ee114012a admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:48.592295 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e37aa-8502-4746-9fd3-170ee114012a admin admin] PATH_INFO: `/v3/roles` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:48.592931 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880e37aa-8502-4746-9fd3-170ee114012a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:03:48.593944 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: WARNING py.warnings [None req-880e37aa-8502-4746-9fd3-170ee114012a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 11 15:03:48.593944 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: warnings.warn(msg) Sep 11 15:03:48.593944 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: Sep 11 15:03:48.594848 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880e37aa-8502-4746-9fd3-170ee114012a admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:03:48.607045 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 754/1499] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 15:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:03:48.611595 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc848762-1679-4aaf-84e9-5239fad2bf75 admin admin] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:03:48.613545 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-cc848762-1679-4aaf-84e9-5239fad2bf75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:03:48.625794 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc848762-1679-4aaf-84e9-5239fad2bf75 admin admin] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:03:48.651236 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:03:48.722219 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a265e-5be1-4986-85ec-2f7eee73d5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b748811b1c8a461188c96c2e314bf507', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:03:48.722853 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a265e-5be1-4986-85ec-2f7eee73d5ca admin admin] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:48.723139 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a265e-5be1-4986-85ec-2f7eee73d5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:48.723407 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a265e-5be1-4986-85ec-2f7eee73d5ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:48.724039 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311a265e-5be1-4986-85ec-2f7eee73d5ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:03:48.725071 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: WARNING py.warnings [None req-311a265e-5be1-4986-85ec-2f7eee73d5ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 11 15:03:48.725071 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: warnings.warn(msg) Sep 11 15:03:48.725071 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: Sep 11 15:03:48.725964 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311a265e-5be1-4986-85ec-2f7eee73d5ca admin admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:03:48.738259 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 746/1500] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 15:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:03:48.742607 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880e37aa-8502-4746-9fd3-170ee114012a admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:03:48.744166 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-880e37aa-8502-4746-9fd3-170ee114012a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:03:48.778462 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880e37aa-8502-4746-9fd3-170ee114012a admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:03:48.877716 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf38ea-bf68-4d2b-b44c-97a230c8bb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b748811b1c8a461188c96c2e314bf507', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:03:48.878370 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf38ea-bf68-4d2b-b44c-97a230c8bb79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:48.878644 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf38ea-bf68-4d2b-b44c-97a230c8bb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:48.878950 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf38ea-bf68-4d2b-b44c-97a230c8bb79 admin admin] PATH_INFO: `/v3/projects/e5eda41ec9674f42a8adba9dd2cd5758/users/c6b4653f8b28463d8bea122c6a065715/roles/c9da950c75da4939b8bd649c11015650` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:48.901097 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf38ea-bf68-4d2b-b44c-97a230c8bb79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5eda41ec9674f42a8adba9dd2cd5758, user_id=c6b4653f8b28463d8bea122c6a065715, role_id=c9da950c75da4939b8bd649c11015650)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:03:48.902163 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: WARNING py.warnings [None req-56bf38ea-bf68-4d2b-b44c-97a230c8bb79 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 15:03:48.902163 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: warnings.warn(msg) Sep 11 15:03:48.902163 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: Sep 11 15:03:48.904154 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf38ea-bf68-4d2b-b44c-97a230c8bb79 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:03:48.929202 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 755/1501] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 15:03:48 2022] PUT /identity/v3/projects/e5eda41ec9674f42a8adba9dd2cd5758/users/c6b4653f8b28463d8bea122c6a065715/roles/c9da950c75da4939b8bd649c11015650 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:03:48.942283 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271f5e3-4854-4179-8fa7-c81f1e3eab37 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:48.942572 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271f5e3-4854-4179-8fa7-c81f1e3eab37 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:48.942892 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271f5e3-4854-4179-8fa7-c81f1e3eab37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:49.000327 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-f271f5e3-4854-4179-8fa7-c81f1e3eab37 None None] MFA Rules not processed for user `c6b4653f8b28463d8bea122c6a065715`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:03:49.001542 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-f271f5e3-4854-4179-8fa7-c81f1e3eab37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:03:49.078657 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 747/1502] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 15:03:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:03:49.091609 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b06f9-750b-49f6-b112-ca2bdbb6a333 None None] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:49.091900 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b06f9-750b-49f6-b112-ca2bdbb6a333 None None] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:49.092892 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b06f9-750b-49f6-b112-ca2bdbb6a333 None None] PATH_INFO: `/` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:49.094237 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 756/1503] 10.222.0.33 () {54 vars in 964 bytes} [Sun Sep 11 15:03:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:03:49.099072 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5fd51-ee29-4b26-9e9c-196b1cc9e14e None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:49.099364 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5fd51-ee29-4b26-9e9c-196b1cc9e14e None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:49.099640 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5fd51-ee29-4b26-9e9c-196b1cc9e14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:49.131380 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:03:49.175422 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-91b5fd51-ee29-4b26-9e9c-196b1cc9e14e None None] MFA Rules not processed for user `c793eafcdba249a5af3ab948e9774367`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:03:49.176637 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-91b5fd51-ee29-4b26-9e9c-196b1cc9e14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:03:49.254539 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 748/1504] 10.222.0.33 () {58 vars in 1061 bytes} [Sun Sep 11 15:03:49 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:03:49.262316 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b06f9-750b-49f6-b112-ca2bdbb6a333 None None] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:03:49.263936 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-151b06f9-750b-49f6-b112-ca2bdbb6a333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:03:49.317172 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:03:49.371163 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b06f9-750b-49f6-b112-ca2bdbb6a333 None None] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:03:49.446591 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae984b0-af95-487e-932c-cb1616328a3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c793eafcdba249a5af3ab948e9774367', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:03:49.447231 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae984b0-af95-487e-932c-cb1616328a3d service cinder] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:49.447512 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae984b0-af95-487e-932c-cb1616328a3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:49.447782 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae984b0-af95-487e-932c-cb1616328a3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:49.449368 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6ae984b0-af95-487e-932c-cb1616328a3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:03:49.549404 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae984b0-af95-487e-932c-cb1616328a3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:03:49.550764 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae984b0-af95-487e-932c-cb1616328a3d service cinder] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:03:49.554241 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:03:49.570376 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 757/1505] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:03:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:03:49.615924 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963d8ef-c784-4201-ae8e-a5c63725d4fe None None] REQUEST_METHOD: `GET` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:49.616382 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963d8ef-c784-4201-ae8e-a5c63725d4fe None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:49.616484 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963d8ef-c784-4201-ae8e-a5c63725d4fe None None] PATH_INFO: `/` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:49.617703 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 749/1506] 10.222.0.33 () {52 vars in 830 bytes} [Sun Sep 11 15:03:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:03:49.621202 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae984b0-af95-487e-932c-cb1616328a3d service cinder] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:03:49.622420 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-6ae984b0-af95-487e-932c-cb1616328a3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:03:49.721465 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae984b0-af95-487e-932c-cb1616328a3d service cinder] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:03:49.814528 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0fbd60-a865-45c9-86c8-415dfaa8f734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b748811b1c8a461188c96c2e314bf507', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:03:49.815184 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0fbd60-a865-45c9-86c8-415dfaa8f734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:49.815380 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0fbd60-a865-45c9-86c8-415dfaa8f734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:49.815646 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0fbd60-a865-45c9-86c8-415dfaa8f734 admin admin] PATH_INFO: `/v3/users/c6b4653f8b28463d8bea122c6a065715` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:49.829235 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0fbd60-a865-45c9-86c8-415dfaa8f734 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6b4653f8b28463d8bea122c6a065715)` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:03:49.831282 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: WARNING py.warnings [None req-1d0fbd60-a865-45c9-86c8-415dfaa8f734 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 15:03:49.831282 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: warnings.warn(msg) Sep 11 15:03:49.831282 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: Sep 11 15:03:49.833351 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0fbd60-a865-45c9-86c8-415dfaa8f734 admin admin] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:03:49.844868 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=100 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:03:49.900433 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-1d0fbd60-a865-45c9-86c8-415dfaa8f734 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6b4653f8b28463d8bea122c6a065715'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:03:49.913960 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-1d0fbd60-a865-45c9-86c8-415dfaa8f734 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6b4653f8b28463d8bea122c6a065715'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:03:49.937963 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-1d0fbd60-a865-45c9-86c8-415dfaa8f734 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6b4653f8b28463d8bea122c6a065715'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:03:49.947551 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-1d0fbd60-a865-45c9-86c8-415dfaa8f734 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6b4653f8b28463d8bea122c6a065715'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:03:49.947984 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.notifications [None req-1d0fbd60-a865-45c9-86c8-415dfaa8f734 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6b4653f8b28463d8bea122c6a065715'} {{(pid=81430) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:03:49.953980 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 758/1507] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 15:03:49 2022] DELETE /identity/v3/users/c6b4653f8b28463d8bea122c6a065715 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:03:49.958405 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4d874-f823-4aa6-b412-9671a5bbc160 None None] REQUEST_METHOD: `POST` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:49.958688 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4d874-f823-4aa6-b412-9671a5bbc160 None None] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:49.958995 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4d874-f823-4aa6-b412-9671a5bbc160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:50.016041 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.auth.core [None req-1cc4d874-f823-4aa6-b412-9671a5bbc160 None None] MFA Rules not processed for user `53878dbb519c4b3d8858375928f1c2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81431) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:03:50.017252 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-1cc4d874-f823-4aa6-b412-9671a5bbc160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:03:50.092690 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=100 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:03:50.105116 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 750/1508] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 15:03:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:03:50.115563 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0fbd60-a865-45c9-86c8-415dfaa8f734 admin admin] Authenticating user token {{(pid=81430) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:03:50.116804 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-1d0fbd60-a865-45c9-86c8-415dfaa8f734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:03:50.224801 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0fbd60-a865-45c9-86c8-415dfaa8f734 admin admin] Validating token access rules against request {{(pid=81430) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:03:50.279236 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:03:50.301035 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb0c17-5258-434d-bd0b-d2e1449309d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6eb236e80b4395a7aa77020b2f3a4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e4a3d44808e4b4092cfc4ee284c0176', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81430) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:03:50.301650 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb0c17-5258-434d-bd0b-d2e1449309d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:50.301928 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb0c17-5258-434d-bd0b-d2e1449309d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:50.302203 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb0c17-5258-434d-bd0b-d2e1449309d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81430) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:50.303831 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.fernet_utils [None req-86fb0c17-5258-434d-bd0b-d2e1449309d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81430) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:03:50.412625 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fb0c17-5258-434d-bd0b-d2e1449309d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:03:50.413982 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fb0c17-5258-434d-bd0b-d2e1449309d6 service neutron] RBAC: Authorization granted {{(pid=81430) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:03:50.432269 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: [pid: 81430|app: 0|req: 759/1509] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 15:03:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:03:50.795286 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4d874-f823-4aa6-b412-9671a5bbc160 None None] Authenticating user token {{(pid=81431) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:03:50.796594 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.fernet_utils [None req-1cc4d874-f823-4aa6-b412-9671a5bbc160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81431) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:03:50.808023 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4d874-f823-4aa6-b412-9671a5bbc160 None None] Validating token access rules against request {{(pid=81431) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:03:50.911429 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92e96e-86a1-4a86-9700-94670f114625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53878dbb519c4b3d8858375928f1c2d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b748811b1c8a461188c96c2e314bf507', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81431) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:03:50.912090 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92e96e-86a1-4a86-9700-94670f114625 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:50.912367 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92e96e-86a1-4a86-9700-94670f114625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:50.912639 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92e96e-86a1-4a86-9700-94670f114625 admin admin] PATH_INFO: `/v3/projects/e5eda41ec9674f42a8adba9dd2cd5758` {{(pid=81431) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:50.922517 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a92e96e-86a1-4a86-9700-94670f114625 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5eda41ec9674f42a8adba9dd2cd5758)` {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:03:50.923508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: WARNING py.warnings [None req-0a92e96e-86a1-4a86-9700-94670f114625 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 15:03:50.923508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: warnings.warn(msg) Sep 11 15:03:50.923508 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: Sep 11 15:03:50.924723 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a92e96e-86a1-4a86-9700-94670f114625 admin admin] RBAC: Authorization granted {{(pid=81431) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:03:50.983904 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [None req-0a92e96e-86a1-4a86-9700-94670f114625 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5eda41ec9674f42a8adba9dd2cd5758', 'request_id': 'req-0a92e96e-86a1-4a86-9700-94670f114625'} {{(pid=81431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:03:50.997111 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [None req-0a92e96e-86a1-4a86-9700-94670f114625 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5eda41ec9674f42a8adba9dd2cd5758', 'request_id': 'req-0a92e96e-86a1-4a86-9700-94670f114625'} {{(pid=81431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:03:51.003961 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [None req-0a92e96e-86a1-4a86-9700-94670f114625 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5eda41ec9674f42a8adba9dd2cd5758', 'request_id': 'req-0a92e96e-86a1-4a86-9700-94670f114625'} {{(pid=81431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:03:51.029778 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [None req-0a92e96e-86a1-4a86-9700-94670f114625 admin admin] The token cache is being invalidate because project e5eda41ec9674f42a8adba9dd2cd5758 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81431) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:03:51.030097 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [None req-0a92e96e-86a1-4a86-9700-94670f114625 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:03:51.030496 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG keystone.notifications [None req-0a92e96e-86a1-4a86-9700-94670f114625 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81431) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:03:51.031985 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: [pid: 81431|app: 0|req: 751/1510] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 15:03:50 2022] DELETE /identity/v3/projects/e5eda41ec9674f42a8adba9dd2cd5758 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:04:00.430068 dv-cinder-856971-1-iscsi devstack@keystone.service[81430]: DEBUG dbcounter [-] [81430] Writing DB stats keystone:SELECT=62 {{(pid=81430) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:04:01.027832 dv-cinder-856971-1-iscsi devstack@keystone.service[81431]: DEBUG dbcounter [-] [81431] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81431) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}